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the sake of succinctness, mascurine. pronouns are used 
throughout this paper, but are intended to *ref er toboth females 
and males. Having considered and discarded various clumsy alter- 
natietives,' however, we settled on this "disclaimer** • . • only 
/o point up the problem. 
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. • Preface ° 

The Military Training Ci?ntiers (MTC) group supports the efforts 
of authors using about 100 terminals funded by the Advanced Research 
Projects Agency (ARPA) . These authors are geographically distri- \ 

bated around the United States at fifteen r^ote sites. The MTC grou^ 

. I • 

provides a number of services to these sites; lesson review is among 
tiie services offered. 

The authors of this book assume the reader is familiar with the 
PLATO system' and the TUTOR language.. To m^ke this book as useful as 
possible, we have used specific cases wherever applicable. It was 
written to service the needs of project administrators, .authors, and 
rev^iewers. We hope fchis book will convince project leaders to provide 
review serviced, will persuade authors to seek reviews, and will serve 
/as a handbook for reviewers. Realizing that re^ad'^itg' time is a scarce 
commodity, we suggest t^ following strategies for economical reading 
, of th/s book based, on the^rWier's r^ole and interests Chapter I shpuld 
be read by everyone; in addition, project administrators s^^ 



read sections 5, 7-13, 15; authors 'should read sections 3, 6V 8,/ 10, 11, 
14: reviewers should read sections 3-6, 14, 15, and^chap'teTs^ W^^^^^^ 

We wish to acknowledge thel help of many people In the prepara- 
tion of this book. A special thalks goes to Jim Krakower and Susy Soung 
who kindly supplied samples of th^ir reviews for inclusion in this work. 



Thanks are also due to the rest of the MTC staff for .reviewing this 

\ ■ ■ ■ I ■ 

manuscript. Many-members of the CERL staff (especially R. Allen Av 
Pauline Jordan, and David Meller) carefully edited this book and 



iv 



suggested ';sLgni ficant clar if icatioti to the ideas presented here. We 



are grate^ful to the remote site ARPA/PLATO authors (particularly those 
at Aberdeen Proving Grounds and Sheppard'^Air Force Base) who have coope 
ated with the MTC group during the development and codification of the 
review process. 

Pat Thomas, not only typed the manuscript, l)ut contributed many 
invalua'ble suggestions on both format and content, 

^. Finally, we would like to thank Rosanne Francis and Jerry 
Sweeney for their constant encouragement. 
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Chapter ! - Lesson Revib*/ 

A lesson review is a set of comments about a lesson prepared 
by someone other than the author as an aid for t^vising the lessop' 
to improve its instructional effectiveness. A review include 
sug'gestions about the programming, instructional design, and/o^r content 
accuracy. It entails considerations as' major as the organization or the 
teaching strategy and as minor as grammar or punctuation. f 

This chapter is designed to give you a very brief overview of 

, . ? 

what a lesson review is so that you have a general backgroiind for 
chapter 1% which describes how tb^^ review a lessen, A more pdmplete 
and specific view 'of lesson review is found in chapter III. 
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1 - VyHY HAVE LESSONS REVIEWED? 



Brletly, here are three reasons for having a lesson reviewed. 

f' ■ . ■ 

Few people are^ 'p^erfectly balanced in the skills .needed 
to vrrite a le»^on (e.g., subject matter expertise, 
programming ikill, instructional design). Everyone 
can use son/e help. * . 

Even ano extremely well qualified author loses his 
objectivity about a- lesson after working on it for 
many hodrs and days. Though studait data is useful 
for pointing out problems and new perspectives, ^ j 

there /are some^ types of .difficulties that student 
data will seldom reveal. / 

'9 

Veedback base??! on current or recent lessons produced 
seems! to -be an efficient and useful author training 
technlique^ Presently, only limited instructional 
material directly ''related to topics such as instructional 
design fc/r computer-based education (CBE), is avai:lable~~ 



\ 

\ t 
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2 - VMAT KINDS OF REVIEWS CAM EE PERFORMED? / 

We distinguish between three types of reviews: subject 

matter, coding, and instructional design. The categories over- 
lap slightly. 

SUBJECT MATTER REVIEW ^ ■ . J . 

A subject matter review is concerned primarily the* 
content accuracy including terminology, drawings^ concepts, 
etc. Of necessity, only a subject matter expert can do this' 
kind of review. This type of review will not be treated 
explicitly in this report, although many of the ^comments are 
relevant. 

o f-, " ' 

■ . / 

CODWG REVIEW 

■ ■ ■ / ' , ■ . 

A coding review examines the efficiency, accuracy, and over- 
all quality of the programming. This also- includes the com- 
pleteness documentation such as the use of -define-, sen- 
sible /unit- and -area- names, and other similar considerations 
often called '^software engineering.*' Depending, on the kind 
^ Of programming involved, a coding review may or may not be 
warranted. We feel that occasional coding reviews can upgrade 
the programming capabilities of newer auLhoxs. This topic is 
dealt with only briefly in this book. 



" The th4rd kin4 of a review is known as an educational or j 
^ instructiv-iial design review. This is the type of reviewin ^ j 
which the.MTC group is most interested and^ experienced . In / j 
this paper when the term ''review'* is used without specif yir^^ J 
type, we are ref,erfcing to instructional design review, • 

A standard' though' relatively minor element of an instructional 

\ ■ •■ " ' 

design review is the detection and correction of technical errors - 

errors in grammar,' punctuation, spelling, ^^tc. Some course- 

' ware development groups have staff members whose duties are . 

nearly identical to those of an editor isES*^ assistant. 

Another part ,of the instructional design review seems almost 

Jo ' . . . ■ S*7 

■ ' q ■ 

as if it should be part of the coding review; When reviewing, 
we typically find it neqessary to examine the coding for the 
judging or answer processing. Errors here can mean inacces- 
sible help sequences,* improper feedback, and misjudged responses. 
For example, we once found' the following coding: 
answer ,375 

wrong 3/8 ' ^ 

write Your answer must be in d^imal form. / ' 

Programmed in this way, the author *s program happily accepted 
the fractional form of the answer. Unless he- collected and ex- 
amined the *'ok" responses for some student who happened to use. 
the fractional form, the author might never discover his eriforl 



Examining the code is the easiest way to detect this kind o^ 
problem. ' ^ --^'^^ 

Details about other concerns of a reviewer are the moi^t 
important and yet the most difficult t<y categorize. Chapt'^r 



n expanck the Jc^l lowing brief \list of considerations : 

• ^)rerequ-isites 

• objectives versus test/i\tems 

• -sequencing 

• tone 
2 • • interaction 

• queji^ioning 





d^lsplays 

And ividuc/lization 



y Instructional design reviews may be don^ during lesson 
devel^^pment or after the lesson has been completed,' 
, IrivProgress "Reviews 
. ' \ 1" ge'i^eral^ we prefer to review a les-sotu'several 
t^^mes during its devj^lt)!^^ This avoids the problem 

of \the author \|pt£L^tn-gW^r along ttiat the reviewer's 
-^OTim^nts cause i/t^^coftstmrtTiK^reWlpions, • This type of 
reviewing also/ytedutes khe amouat /of - criticism an - 
author rfce.lv^s at .o|e time. Mt^n the initial pl|n'I," 
;'tbe objel:\^iy/feL the^'^^ritWriOT^ t^st^ and each se^ent 

_ .yjh ' \ . /'. ■• _ 

ot the les^pxj are re^iew^d successively. In-progresS ' 



reviewing. ipj':^discus's'^avi^^^ in chapter V.^ 



EndrofrLess6n Reylm/^s,.^^' 



Although ik-progre^^ reviete ar>€' generally pre- 



^./^ferable* sci/ne times they ^re ' WtV^e^^lfele . Until 



an 



author has ^xrohf idei^e>.in th^ revieweit-^rn^vhas a working. 
" knowledge of what a -t^vieW;^ is,, supposed to do,! it Is , 
difficult to c^^ince hinv %o submit an unfinished ^: 
•lesson for review. So, soufe end-of-lesson reviews are 



stepping st^nes^ to future in;-progress rev^iews. , 

In-prbgress reviews are only possible if thej lesson 
is in the process of . being developed. In several instances 
this is^not the case. The author may have\ finished the' 
lesson some time ago,, the lesson may belong to someone 
who has left the PLATO system, or the lesson may be , 
-"^^ borrowed from another curriculum developrnjent group in 
order to revise it for a different audience. In each 
of. these cases, an end;-of -lesson review is appropriate, 
o Details about the special needs and procedures for this 

kind of review are given in chapter IV. 
One last admonition: one might tend to view the distinction ' 
between end-of-lesson and in-progress reviews as summative' versus 
formative evaluation. ' Though there may be some parallels, .this 
Comparisdn is basicrlly ..inaccurate-, ' Both types of review are 
formative; both prei^silppose'tnodif ication of the lesson materials 
following the review. = ' ' ^ 



\ ■ 



1 , ■ 
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Chapter ri -TERFORNiiifi A Lesson Revi^^ 

Much of -tiie material contained in the following sections was . 
influenced by Lesson Review; Some Suggestions , which was developed by - 
Merle Goldstein as part of a project for her Master degree while she 
was °wprking rwLth MTC. Chapter II, in particular, with some minor changes- 
has been taken directly from Lesson Review . . 

The following -chapter was designed to be used to review any • 
packaged lesson or lesson segment. A packaged lesson is one, that is 
basically free-standing,' self-presenting, and capable of . being admini- 
stered identically each performance. Programmed instruction , CBE, and 
films have all been "reviewed usihg the techniques described in this ichap- 
ter. It is implicit in this and successive chapters- that tfie author is 

not present while the reviewer is going 'through the lesson. This both 

' ' \ ' ' . >■ , ' ' 

saves the author time and prevents situations in which either the author - 

feels^he must defend- the lesson, or ..the review^'s objectivity is hampered 

by the" author ' s presence. ' - ^ 



3 - HOW IS THE REVIEWING TASK STRUCTURED? ■ ' ^ 

Part of the difficulty in preparing a review is deciding its 

focus. What kinds of coniments are most useful? Where should one 

put the most effort? * 

'It is a little easier to^ answer these questions . if one has 

nri organized "picture" of the different kinds oit 'C^^ it is 

* ■ . ' ' ' ■ , ^ ■ ' . ■- - 

possible to make.^ One useful way of thinking abqlit:' these different 

kinds of comments visXto imagine them a§ runnipi^ along a continuum 

- . '■ / ' /' . " ■ 

•goint'?from those .eoncerrfed with the f undagt^i^al structure of the 
lesson to ' those concerned with the lesson ' s- "polish. *■ These comments 
can be thought ^f as belonging on four different levels: 



Beginning ^—f 1- ' | - 1- — 1 End 

: Planning Design Implementation Polish 



LEi/El ONE, PLAiUyiWG ' 



G^omeuts^ on Xhis level' concern preliminary fundamental decisions 

^ ' ■ 7.^-.. ' " ■ "•■ . . ^ . - 

made during the planning stages of a lesson. For (example: 

• The intentions, goal ^ objectives of the author — 

th^ purpose of the le,sson \ - . ^ . 

'm: Assumptions made about the student' s general ^ability 
and Specif icvbackground in the subject • / 



• The rielationship between a particular lesson and tlie, 
^= . , course or curriculum of which it is a^^S^t 



• "^Exactly what cbn-tent to include and exclude from the 
lesson — what content should be selected 

• • ' - 
" 11 • ' ■ 



^' ' ^ \ ■ ■'■ , ■ ' ■" ' • ' ■ , \ '- A 

Comments on this level concern decisions made about the de- 
sign tyi the lesson. For example: 

'* # The choice of particular teachirfg stratej 

• The selection of media 

• The way the lesson will>a:ndle students with hetero- 
geneous background^^-^d abilities ~ the extent of indi 

s, vidua lizatiojv-^^nd^^iH.fferential routing ; 

• The oj^ganlzation of the^ntent 



Comments on jthis level are concerned with how well the pre- 
vious decisions dre carried out. For example : 

• How w^lL tte 'chosen teaching strategies and media are 

b^^ing employed • ^ * ' 

' *. . 

; , • ;the lessdn handles Individual differences 

in corrective, feedback and remediation 

- • The appropriajteness of the lesson's tone and style* 

/- • The le,sson test, if any 

- The lesson flow -- e.g., how smoothly frame-to-^frame 

and tdpic-tq-topic transitions are made ; 

• The equality and quantity of student interaction 
^•g*j is enough practice provided, is there sub-- ' 

^ stantial material the student is not actually required 
to use' in some^way during the lesson * 

• The appropriateness and quality of the questions — ^\ 
; ; '^^•g-j are they aimed at the proper skills; ^re they 

f technically sound 

' ■ ' ' [ . 'V ' , 

• The cldrity, of the explanations, the appropriateness of 

the reading level, the illustrations, ^nd th^/exlmples 



LEVEL FOUR r- P0LISHIWG/FIWI5HIWG ' \ | 




Comments oh thlis level concern editorial, aesthetic ,~\ and 

\ ■ ' . *■ . •Hi 

production issues. For e:cimple: \ 

• The language. — e.g. , grammar, spelling, typograpfir 
ical errors, etc. • - 

• Consistency in the use of terms, instructions, keys, 
" etc . :• ; . 

• The technical quality of the lesson as a graphic pro-r 
ductibn e.g. margins, spacing, indentation, 

' accentuation, etc. 

• The aeschatic appeal, of the lesson 

*Ohe of "the first things a reviewer m.ust do' is decide ^how many 
inJ which of these levels he will include in the review. 

Jn general, a good rule is tp concentrate first on prob- 
lems at the: more basic , structural levels (that is, levels one, 

-^Two, and three). Not until after the problems'^at these, leviels are 
c^edrfed Cip iB-t^ere much point in dealing with ^problems at 
level four. ,One reason for this rule is that if the reviewer is 
questioning something at a relatively basic Ifevel, there is 
less need (or time) to deal with the problems falling into level 

. four. For example, if one doubts the neeci'for a certain segr 

' menrt pf the lg.sson, there is nothing to be gained by pointing 

out that a worj^ within tbap segment, was misspelled. An ex-. 

> , ■ f. 

ception to this rule is that when a. specific kind of problem 

» ' ■ ■ • • . ^ 

crops up again and again within a lessorf^ it i^ helpful to point 

that out to* tfie author nO matter at what level the pfoblem 'occurs. 

For example, ft a word is consistently misspielled it ' would be 

worthwhile to mention, . - . 



Finally, it is important to note that this lev*el^ approach 
becomes inopera^tive when the author, of a lesson specif ically .re- 
qui^sts comments of a particular kind — unless/the reviewer is 
a position of authority that allows him to insist upon the-^ — 



in 



neeid for. comments at all levels, 



/ 



/ 



^ - VilAT ARE USEFUL REVIEWING PROCEDURES? 



mLTJPLE PASSES 




It is horribly inefficient to look simultaneously for all 
-^t-tfe -dif ferent kinds of problems a lesson might contain. Des- 
cribed below is a procedure that is more efficient. 
First Pass • 

Get a general impression of the lesson identify 
for yourself what your biases are concerning the lesson 

Set up "flags'* as to where (at what levels) you are 
going to fqcus your attention. 

Take ;notes concerning what goes :,hrough your head 
as you' work your way through the lesson. Write down ; 
everything , (Remember', first iinpression-s occur only 
once) , . ' , ' ' , 



sions occijr only once. 



Second Pass* 



Note your perceptions as to the objectives of the 
lesson, V * . * ' '5^ , 

Note your perceptions ^ of ^what assumptions the author 
is making conderning what student^'' already know. 

Take brief notes on all the information (i.e., the " 

lesson content) '^that the* author presents to you — 

or rather to i.the^'"student^';jf/hose role your are simulating 

Work through the lesson as both a "smairt" and a 

■ ■ ■ , " ~ c 

"slow" student in order to see how the lesson handles 

---- - — . ^ 

.both, / ^ ^ 

Note your perceptions as to the organization of the 
lesson, ' Thi$ is', the time to make -an outline of how 



.tlu> ideas T4ow and to attempt to depict the branching, - 
This is a roa^mp of the lesson and should be compared 
to the flow chart x^f ideas that the author supposedly 
sent along when he requested a revi^gtST^ They, of course 
should match exactly. That is, thef ideas in the lesson 
along with the ax^ailable heI>D and rJ|jview sequences 
should be exactly as the authorxiias claimed. 



Third and Additional Passes 




• Use these passes to take a closer look specific 

segments about which you have questions an^ if appro- 
priat^j^^^J:^ look at the coding of answer judging. If 
)u are working through the lesson with the prirvbout, 
follow the flow of the lesson on the printout as yoi 
go through the lesson. 

Having gone through the lesson and taken notes concerning 
^your reactions to it, you are- now ready to synthesize your re- 
; marks and to organize all your material in order to prepare ar t 
* I report for the author. ' 
; -I Depending on your style and the strategies , employed by ^ 

the lesson, your^written report might include the following 
p^ts: 



letter or other introduction to the review 

A section devoted to ypur general comments covering / 
the conclusions, you reached based 0n your observa- / 
tions. (which have bee^ documented in the materials / ' 
mentioned above) . Su^h a section eould include ay 
comparison between yj\xt observations and the author's 



7 ■ 
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. *See section 6 for a discussion of what items, an 
should supply the reviewer. , 



/ 




stated intentions as well as a statement concerning 
how consistent the different elements of the lesson 
are. In a. sense, this section would be devoted to 
conments related to levels one and two and some com- 
ments related to levels thrae and four that come up 
■ again and again in the lesson, 

• A section devoted to your specific comments concerning 
particular segments of the lesson — i.e., the de- 
tailed criticisms and suggestions. If -you're working 

- - with a printout, it is generally useful to write 

these specific comments right on the printout itself. 

• Sections detailing information the author failed to 
furnish yop with or which seems incomplete or incorrect, 

- A content outline sketching the subject matter 
^'^ as you perceived it. 

- A list of the assumptions that seem to be 
made in the lesson. 

- A flow chart sketching your perception of the 
' routes a student can. follow ^s he :works 

through the material. - 

- A^list of thie sub,skills which ar^ implied in 
.» the lesson's objectives. 

A . - • . ■ ■ J :■ ■ - ■ 

SUGGESTEV TECHNWLiES 

The techniques listed below can be grouped according to the 
levels of review discussed. Not all techniques are applicable 
to , all lessons, of course. They may, however, serve to get you . 
going, giving you some ideas for developing techniq^ues of your 
own. ^ - 

Level One Planning ^. 

• Objectives . Note the author *s intentions as stated in 



docunuMits, in the lesson, as implied by the title, ... • 
etc. Indicate your perceptions of the student and 
lesson objectives based on working through the material. 
If your percept ions don ' c match the author's apparent 
intentions, question him as to which truly reflect 
his wishes. Make suggestions as to possible changes 
in the stated obj ectives ,or in the lesson. 

• Content Record your perceptions of the content of the 
lesson — in most cases, this would involve writing 
down the information presented to the student. This 
might be prepared as a content outline or as alist 

of concepts. Compare this outline with the objectives 
(as stated by the author and as perceived by you). Note 
which objectives the outline seems to support. Compare ^ 
your content outline with any documented listing of 
intended content as supplied by the author. Note in- 
consistencies between content and objectives (as ob- 
served or as stated by author) . Make suggestions as to 
changes in the content of the lesson.' 

• A ssumptions Record your perceptions of what assumptions 
the author has made about what the students already, 
know and can do. Compare these to a profile of the 
,target student population or list Sf entering be- 
haviors of the students. J?oint out inconsistencies or:, 
i¥ no description of the target population exists, 

call the author's attention . to your list of perceived 
V assumptions to see if he agrees^ with your perceptions. 
If necessary, suggest that a series of diagnostic ' * 
tests be developed to provide some information con- 
cerning students \ entering behavior., Suggest additions 
or deletions of material based on t^e above. ' 

Level Two — Design ' 

•i • " ^ ' ■ ' • 

• Teaching Strategy , Compare the instructional techniques 

i . . / 



(e.g., tutorial, drill and practice^-/ simula- 
tion, inquiry) with the objectives as perceived and 
stated. Note whether they "fit" each oj^her. Sug- ^ 
gest possible' changes in technique and/or reappraisal 
of the objectives in view of the strategies employed. 

0 Sequencing Note the routing procedure used (linear, 
branched, fast-slow tracking, etc. ) . Prepart a flow 
" chart of the lesson. Compare with student profile 

. and flag possible problem areas. Analyze student data 
or advis.e the author about areas that should be watched 
especially carefully when students go through the 
material. Point out those sections where the author 
has used branching most skillfully. 

• Organization The content outline can be used to document 
the perceived organization of the material in the lesson. 
Note whether there is a clear structure and/or a smooth 
flow to the material — i.e., are the relationships 
among concepts clear? Does the arrangement of the sub- 
ject matter seem confusing? 

Note the educational strategy being used in the 
'lesson. Is it consistent with the objectives and//r 
the student profile? Suggest changes and give appro- 
priate compliments . ^ 

Levdl Three — Implementation/Developmen^ - ^ 

• Student Interaction Indicate what^interac tion/is in- 
volved — i.e., what skills the student, n^eds/to success- 
fully interact mth^ the computer: perfect spelling, 
fast typing ability. In some lessons we 'v^^ reviewed 
which used gaming to motivate the studentar to learn some 

' 'rote material, the games were so complex/that- learning 
the rules required a higher skill level/ than did the 



"^lesson material' itself 
' - 's Compare the type 
the ob j ec t ives ,/ content , student^rp* tifkf *1 evel of 



1 

Compare the type and quantity y'f interaction with 



nastery, and the questions on the criterion test (if 
present) that accompanies tfie *'levss.on. -Npte any in- 
consistencies, but reinforce .^ound. practices too. 

: o 

•Outline the relationship between the content 
(information presented to the student) and the inter- 
actions required of him in the lesson. For example, 
note whether the student is provided with all the 
information he needs to answer questions. And note 
whether the student is actually required to demon- 
strate the skills specified in the' objectives. The 
objectives may, for example, specify that the student 
statei a definition; the lesson, however, may only ask 
the ^tudent to choose the best of four Supplied defini- 
tion^. This kind of problem is, ^frequently found. 

/ D e t erMne- 4iow t he a uLho r-liaiiglgs individual" d^ 
ferehces in response to questions. Note how \1ie 
handles feedback and remediation. Do different responses 
evoke different feedback? Do successive incorrect 
responses elicit ever stronger hints? Can a .student 
^go through a lesson without learning the material be- 
cause the lesson always gives "him the answer on the 
third try? ; ^ ♦ 

Note how much practice the student is provided 
and how\it is provided (i.e., massed or distributed) , 
CQmpare the amount of practice with the level of 
mastery to be achieved. Analyze the questions (in 
the lesson and in the test) and, if necessary, note 
violations of good question-writing techniques. 
Norman Gronlund's Measurement and Evaluation in Teaching 
(1971) and Norris Sander^s Classroom Questions (1966) 
are good standards for question construction^ Compare 
the type of question Ujsed with your perceptidn of the * 
objective the author is trying to test at that point. 

Presentation Determine whether the author is using 
the^ instructional techniqueis and the medium 



appropHately . Look at the text as a hook-reviewer, • 

the "animations as a' film reviev/er.'' 

/ Note the overall tone of the production. Are 

.remedial work and feedback condescending? Compare the 

.'cone with the student profile, the context, the ob- 
jectives, the subject area.. 

Note the author's attempts (or lack of them) to 

■ keep the students motivated. Compare with what is 
kiuvu^ about the reasons the students are working through 

;^the^rterlal'^- ^^^^ suggestions as necessary '^fd con- 
gratulate, the author when his lessons seem particularly 

'^^jDro.mising . ■ ' x / 

"N Note the author's attempts at directing p^^^ 
student's attention to important material and,i in 
: ~gel^i7nTl^iIpTHg^^ wltii the— 

material to be masteredv Suggest capitalization, under- 
lining, boldface ietterdng, spacing boxes, or arrows, 
if necessary. Inspect the explanations to determine 
if they are clear. Check tp see if there is aiy jargon 
>. or convention with which the student might be uhfamiliar 
^ Determine whether the 'illustrations ar^ being used" to 
. ^ o the fullest advantage.. 

Level Four ---J^lik^ng/Ed^fishing ♦ \ 

• Detennine wjietlrer the lesson is in good , enough shape , 
to- undergo a final polishing. If not, concentrate' on / 
levels one, two,' and 'three.. If it is, proceed with the 
final editing. Note whether the materials are^^ 
technically satisfactory and whether >the lesson is ; 

• aesthetically pleasing. ' J' 



i'here are csix types of information the author^* finds in . 
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a rev;4w of his lesson. ' • V . " \ . ' ' 

' / ■ . ' ■ ■■\. . ■ .. A' ■ ' 

/ • He finds out what the iNeviewet consider well-done. 

/ ' This is ^an important part^ that' is frequently for- 
,/ , gotten by reviewers. It ^kes the reviewV^re palatalS)le 

/' . increases the likelihood the author*'"S|.l' request ' \ 



a review in the,,future. ... A^. ' \\ 

He finds a list of errors aod' problems which h'^did 



/ ' ' anticipate or, which h^. dealt with unsuccessfHllu. 

/ ♦ ■ . j ^ • ■ • 

• He finds proposed solutions or suggested resources \ 

. ' which c^n aid him in correcting problems. X 

■ 0 ■ . • ■ " S,, 

• He gets'a list of potential problem^ needing more \ 
^^t^' These are items which the reviewer is" uneasy 

^ / about 1)ut lack's the experience or data tt) definitely 

"I - ~ label as problems. - \ ; 

."■^ , . . , • ^ " ,\ ; ' • ,^ ^ • _ i . 

. - gains a perspective as to how his lesson looksS - 

relative to other lessons. ; 

• He finds that the reviewer needs clarif icatipn of ; ^ % 
such items as , the purpose, use,, or d^ign o^/ the \ ' I 
lesson. These can be tesolved during^^ future int^er-l / 
actions with the reviewer. * \ I 



A/ chance for you to practice using these skills is provided 

. ■ ^ ■ ■ : / 

in chapter ¥1, A segment of programmed instruction has beea ' ' 

^" V ' :> ^ ■ - ' . 

•'seeded" with a number of errors which you are to locate. A' 

list of known errors, categorized by le^/el, , "follows the practice 
exercises. ' 
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CwPTCR III - Reviews OF Computer-based iNs-mucTioN Lessons 



This ''cti^^ter describes, special aspects pE reviewing CBE les- 
sons, rationale for lesson standards, how to select a reviewer, var- 
ious us4s for 4 review, how reviews relate to s:udent testing, re- 
viewer responsibiHtles , etc^ 



28 . 
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5 - V*V\T FORM SHOULD THE REVIEW TAKE — WITTEN OR ORAL? 



Some parts of a revlew' should be. written; others are best 
delivered orally. The general coyiments about the lesson should 
be written (preferably typed). 



Comments "should, of \coursr^7^&ekept as part of the docu- 



mentationvof the lesson. )(i written document is harder to igndre 
than oral cpmmentji. One cannot simply forget (selectively or 
unconsciously) sorfe of the more unpleasant suggestions. The 
permanence, of a writl^ record is helpful iiV that the -dut^hor can 
re-read the comments to extract their full meaningj^or he/ can 
show the review to colleagues for theij gp^i1^^^ons. Also, many 
people have a .useful han^;^jLip^fiout discarding^ersonal, typed 
' correspondencer^^ they are likely ju^-file^ it . It's useful fbr 
. the^ author to be able to look at the review a few months later 
wTien^e has. gained more * experience with, the lesson. In retro- 
spect, he may better observe his "blind Spots" or may find the 

■ ■■ ^ . • ' 

reviewer 's perceptions were inaccurate. - 

. • ■ : ^ / ' . 

■ ■ , ' ' \ ^ ' ^ ^ ' ^ ' ■ 

End-of-lesson reviews may be typed and mailed^ in-progress 
•' ' ' , I- ■ ' ^ ... '■ ^3 

reviews must be delivered faster. With PLATO 's aid and a hard- 



-Jiojyi&B-er-prtn€er ^ the ;revieWer''caTr-wrjrfe--hiw roTtrments bn-li^ne 



to Speed communications A hardcopy of the. comments should then 



be obtained by author and reviewer. for a permanent record. 
ORAL ' .^ ' ■' '- , ■ ^ ,. 

■ ^ ' • ■ ' .' ' .' '-.^ " ' : . ■ 

Though' we 'rie biased towards written reviews, we feel there is 
^ * an important/ role for face-to-face or^ on-line communication in 

reviews. Th^ personal interaction can seirve to direct the review, 

. 't 

to soften c^riticisih, and to act either as an introduction to the 
written review or a follov^-up to it.. In a later .section we ex-- 
pand on the Idea that even^; an end-of -^l^sson review should" -not 
be a' one-shot ptocess but. should serve instead as the beginning 
of an interaC/t^Lon. 

RECORVims ' ■ 

The MTC group has considered, but not implemented, one inter- 
esting reviewing format. We contemplated using a stered tape, 
recorder to record the computer- to- termina.1 input in one'' channel, 
while simultaneously recording the voice of the reviewer' in the 
second channel. The, method lacks certain desirable features of '^ 
the written review, but offers. a great deal for lessons which 

^ cpntain complex animations or ^lnvolved simulations^ Only the . 

>. ■ /I ■ . 

difficulty in finding a porJ:able stereo, tape system available ^ 
to both author and revieW^ has prevented implementation. The 
'*talk'' option's monitor feature provides much the'' same^type of 
interaction as the taping method proposed abov6. However, 
^ "talk" lacks permanence and infotmation on tapes is difficult to , 
> index or retrieve. For these reasons, tapes and on-line con- 
versations must be accompanied by Written, comments. 



WiAl" SHOULD TOE AUTHOR PROVIDE FOR- THE REVI.EVER? 



Most of the' t bllowing items should., already be a part of the 
author VsvUOtebook or documentation for the lesson, . 

A; 

• ^ A description of the student population* . ' , • 

• The overall cut-ricular plan and where this lesson 
-fits into it -r- the lesions' prerequisites. 

• "The lesson objective's, xkese, may be dif f erent f rqm 

the student objectives. For example, a lesson objective 
might be tio arouse a student's curiosity about a 
topic (i.e^, a lesson objective may be^ attitudinalK 

• The flow chart for the student, branching in the lesson ^ 
(not a programming flow chart). We feel that a simple 

' ^' ' flow'-chart should be drawn at the lesspn''s inception 

and kept up to date *as -the lesson develops. We 
■ prefer a flow chart made of ^oxes ©r circles connected * 
with arrows showing . brief ly the lesson s content and 
branching. Nothing elaborate is needed. Fpr example,. 
* see page 79^. " • 

• Any handouts, workbooks, worksheets, or supplementary 
material which accompany the lesson. Also the slides 
or audio disks that are used by the lesson, ^ 

The following items are useful, but not. always necessary. 



The lecture notes, text, films, and ottier teaching 
materials which are replaced by the lesson/'or from 
which the lesson i§ drawn, \ 

A variable map if (Heaven forbid) -define-s haven^'t 
been used or^ if the lesson is unusually complex and 
a code review* is desired. 



• Student data, if available^. An on-line copy of the 
datafile i$ very much better than a printout since it 
can be cpmputer-searched. , 

'The auJJKJr should note how the trial students (^iffer from 
the target population (prerequisites, ability, etc.)» Unfor- 
tunately,-* most reviews have 'to be undertaken with only two br 
three of these components present. ^The more items that ^cfan* be 
acquired, the more complete and accurate (and less time consuming 
the review will be, . ^ 

There are several steps that the vjevleweiT^nay take to/ 
make his Job easier. If the reviewer copies the lesson in|;o>one 
6f his ovm lesson spaces, he won't be bothered by continual 4e" 
bugging hy che^ author. To this copy he may add this special 

"imain*' unit: : 

* *< " ■ 
. ' . ^ ■ ■ * . 

unit imdinu ,;. ' ^^ 

at 101 ' 

showa mai'nu $$this shows the name bf the. main unit 

* in the upper left hand corner o£ the 

sciree^n . ^ — 

He can also' insert any extra -term- commands needed to facili- 

.? - 

tate movement within the lesson. The following code is added to 

• • - 

the initial entry unit (ieu): 

imain imainu - . w 

'''f 1;ls t command s , j ump , j umpou t , next , nex tno.w, back , lab , da ta , 

help, end, tetm $$these last two lines shouL 

/ % be added if working with a 

printouT:. - . ■ » 

*list V symbols 

I ■ 

The -list- commands are only neifeded if the reviewer is working 
with a printout. When a-''printoiJt is made, these coiflmands provide 



the data for. the reviewer to construct a student. Branching flow 
cliart if the -author has failed to provide it. 
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7 - IF Ri:VIEWS ARE SO 6O0P^ \W DOESN'T ^ERYONE DO 



OTHER miA 



DeVending on your definition, everyone does rfeviews! Les- • 

r ■ ■ ' ■ _ " ' ^ • / ^ 

son reviews or their analogs are everyi:rhere«x Calling them 

views" is actually a seinahtic problem/ they're teally "edit^^" 

sotnething found in ^the publishing industries. Ih PLATO jargon,. 

however, "edit'* and^ its derivatives have anotheji: connotation aiid 

so^>tne word **ru/iew" has been used. Because of the differences 

•in media, the parallels with the editing proce^ss as practiced by 

the editors of textliooks and movies are not ais*' obvious as one 

might expect. A movie editor may cut and "bpllice but can rarely 

reshoot film. The book editor has an even ijiore restricted medium; 

he cannot control speed ,' flow, or tone.' Bpth'^media^ are linear, 

if"" ** ' ti 

non-interactive, and non-*lndividualdzed. .The. objective of all 

editors (including lesson reviewers),, is^ however , basifeally the 

same: to help the author provide a cleat, coherent, organized ^ 

flow of information in order to impart Itnowletige or achieve an. 

effect.' // 2 i ' , 

* Staniiards are available for 

editdrs of the printe.d page; stan- 

dards f^^r film editors seem to be 

les^ well-defined^ more artistiCi, 

This is.reasonable since motion 

pictures are only 75 years old 

whereas printing is 500 years old; 

.30 




In tfje posf, most PLATO lessons 
wert reviewed only tnfotniolty. 
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Both media have professional editor s; however , they are not 
the people who haxfe written the material/. ' ^ 



THE PAST 



In the past it was not a general practice for PLATO authors 
to 'fiave their lesaons formally reviewed . This is hardly sur- 
prising. During^ PLATO* III days, classes were necessarily sma,ll;- 
few lessons had been written and little was known about the cri- 
teria by which to judge a lesson. Standards were just emerging. 
During those ^days, too, it was. possible for an author to knovr most 
' of what was^ known about writing PLATO lessons. All authoring 
took place in one room. "Old timers" (those who had more than one 
year of experience) "^geiierally , knew all PLATO authors and (amazingly!)" 
' all lessons currently under ^development . With easy, access 
* lesson names and the capacity to stote only two 'to four lessons 

at once, authors waiting for space typidally entertained them- 
' selves, by going through someone else's lessons. In 'this manner ^ 
.they learned CBE teaching- techniques. Furthermore, such authors 
typically gave aixd received unsolicited comments abouf^exe- . 
cution errors ar\d instructional strategies based on this ex- 
perience. ^ . , • 

With the PLATO' system in 'its Infancy, ca,reful,..-expensive * 

/ . ■ 'i'^ • ' ■ ■ 

review's were probably not -as' necessary. The' lessons were exper- 

"'^ ^ . . ■ 

* imental, the audience small, ^ and the system constantly flianging. 



THE VRESBHT 



The implementation of the PLATO IV system brings great 
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opportunity and new' i;;esponsiblllty. The wide geographic distri- 
bution of terminals means that many new users are having to 
rediscover the techniques, tricks, and skills for writing good 
lessons. Justifiable security measures and the formation of 
author subgroups mean that there are fewer chances for the novice 
to see a lesson j^eing produced by an' experienced author; generally 
only finished lessons are exhibited . Because of the prolifer- 
ation of authors and lessons, no one can keep a friendly eye on 
each new author and lesson. When comments on a lesson ^re re- 
quested in ''notes** the respondents are likely to he brief and gen- 
eral. With little personal liability at stake and no guaranti^ie 
of r,eciprocation, there is little responsibility to do a thor- 
ough job. . 

Thus, at present, most reviews are done informally or not 
at ^'11. Some of the. larger curriculum development groups have 
a step in their instructional development process which contains' 
many elements of k review as described by this report. Only 
a few of these grjoups produce any written remarks and most of 
.these deal with Subject matter or coding rather than instruc- 
tional design. It seems that content and coding ''errors*' are 
more easily found, proved, and corrected than are instructional 
design i^roblems; furthermore j peo'ple tend to be less camiliar \ 
with the educational aspects o£ lessons. " 

Compared to those for ihore coiranercial publications, the 
standards for textbooks and other educational media (records, 
tapes, film§) seem to be low. The first editions of many science 
texts (which include matiy figures, graphs, and chemical or math- 
ematical equations) contain st4ggering numbers of errors. ' Some 



are . typographical missing minus signs, decimal points, etc.; 
others are scientific misconceptions. Taken as '^gospel" the 
scientific errors may be \propogated by other authors. For ex- , 
ample. The Journal of Chemical Eduoar^i^f^ -fixiints a ^columB^^ 
^'Textbook Errors." Alltlhdugh an error must appear in more than 
one text to qualify as publishable , this column has virtually be- 
come a regular (monthly) featu]re. 

To summarize: (I) things! like reviews are done formally in 
other media, (2) informal reviews have been done inCBE for 
some time, (3) CBE isvconsiderably newer tjhan other media, and 
CBE standards are less well established at this time, (4) the ■ 
amount of energy devoted to reviewing shbuld" reasonably depend 
on tlie type of use and the size of .^he audience, and (5) the 
quality of other educational media is not consistently high^. 



ERIC 
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3 - WY MUST AN OUTSIDE AGENCY OR PERSON LOOK AT THE LESSON? 

* - . ' ' " • 

An" outsider can be more objective and may have ^mo re re- 
viewing expertise than an author or a co-developer. 

OBJECTIunV ' 

Initially it would seem that the author always knows 

/ ' ■ • " ^ ' >' V ■ . . 

best — he conceited the^ lesson, he can feel the objectives at 

• . • ■ / .• ^ . 

a gut level, he. knows every twist, turn, and subtlety in the 
lesson. He has tried and discarded various approaches, stra- 
tegies, and test items.' How could a reviewer have any useful 

comments uj;iless he tS^^d undergbtfe-the trials and tribula- 

tions of developing that lesson? Why not let students find^ 
the^error? ■ ^ ' . 

Overdramatic? A bit. But some authors havie exactly this 
reaction. In accomplishing what is a ratjier difficult ta&k — 
giving birth to a lesson — they may have become so involved in 
the lesson that they have lost their objectivity. For ex- 
ample, an. author may begin to t/rlte. the lesson to follow, the 

" i 

style in which hie goes through it^.Tb a great 'extent the author 
loses his objectivity about details large and small by '^wearing 
a path" through the. lesson. He always puts in the same right and 
wrong answers and thus loses his ability to "proof read" the 
lesson for various student responses. In this way, a PLATO ^ 
lesson is somewhat like a toy which "works" only the :first time ' 
each person tries it (e.g., the "dandy. can" full of springs):, the 

' ' 3*4 



designers or other people who know "the answer'' can^no longer 
perceive and react to a lesson as a Stt^dent would. 

Though one of the advantages of a; team authoring approach is 
the greater objectivity of members qf :the team relative to each . 
other, team authors may also lose their perspective, hence, when 
it comes time to review .the lesson, an^^outsider*' niay be neces- 
sary. To remain an outsider, the reviewer must, at first, be 
ignorant of the author's ^pproach^and . strategy • In general, the 
reviewer shotild be able to approach the lesson vety much like 

a student would by "knowing" only as much as. the lesson -pre- 

■- ' . . ' . * * ■ 

requisites specify. 



EXPERTISE 



In addition to the fact that a developer may not be ob- 
jective about his own lesson, another person^^may have greate?: 
reviewing expertise. Aciording tg the experience of J^lie.. MTC - 
group, one migl)t say a "good" reviewer Is both^^tirrTand made., 
All reviewers seem to develop continually and substantially with 
experience. Thus, there may be someone to wl:\om the developers 
can turn who would be expecially willing aAd/or able to review; 
a lesson. The subject of who should teview a lesson is con- \ 
sidered in section 11. j . 



D - l^^^AT CAN YOU DO WITH A REVIEVi? 



EVALUATION- 



-' As ah;,^^yministraeor you might use the lesson review, as ^ 
an evaluationV You could^use it to decide to hire or promote 
an author, to use or discard a lesspn, etc, Tiiat's not really 
the purpose of the revi^w^ process as we have out^l^^ned it. It is 
very dangerous to use a device designed to do One job to satis- 
fy a different purpose. We would agree" that many of the tech^ 
niques and guidelines given in this paper would be useful for 
developing an evaluation of curriculum materials. However, ' 



several ^problems ^arise 'if a 'Critique is to be used both aiS- a 



review and as an evaluation, " , - 

If the author realizes he Is' going to be evaluated^ by a les- 
son review,* the usefulness of the review' is largely nagatad, -In 
order- to obtain a satisfactory evaluation, the author is forced " 
to put his lessons in a complete anid polished jEorm before asking 



€of the review. By the time the lesson is polished, the author 
has a great deal of ego invested in the lesson and he may find it 
hard^to accept suggestions^ to change it. He is forced, to be 
defensive with respect to the ^reviewer and his cooperation is 
likely to be dependent on how. he feels the evaluation will come 
out. On the other hand, the amount of cooperation he provides inay 
affect the evaluation. In general, he h^s a conflict of In- 
terest. .He can't relax with the reviewer, nor can he request the 
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i ^ribvlew viti^TL it is mosjt needed at an early, urifinlshed 
state --.whfen the lesson is full of untesolved problems, 
"^^---l^^ther.^he mu\t muddle through alone and then^def end his solu- 
tions. \ . ' 

REVlSWhl . ' 

-? _ ■ ■ . 

]- -How about using a review, as a tool for revision of the lesson 

-to improve its effectiveness? This is exactly what a lesson re- 
view is fart It should be designed so the author can choose the 
items which he and the reviewer agree on and go directly" to the 
lesson and in}.plement the changes, 'In\fact, with a review and a 
well documented lesson, any programmer should be able td^llegin 
revision with little delay* • - 

A'lesson review -may also be useid to revis# a lesson written » 

■ % ■ ■ . 

^ for a different audience or with different objectives. Such 

t.^ . " • ... 

a case may arise when a course is re-designed or when a lesson is 
boi'rowed from anothei^;.^institution. 



TRAIWIWG 



Lesson reviews have been put -to good use as training de- 
vices. By examining reviews of other lessons, a new-- author ' 
can learn'what typical sorts of problems he sliould avoid. A 
number of authors have found this to be a more convincing and 
less tedious way of learning many - instructional design prin- 
ciples compared to reading about contrived examples or perusing 
lists of ^'do's^Land "don'ts.'' - ,/ 

It is.^ to everyone's advantage for the head .author at a 

■ - ■ ■ ' ■' > , ' ^ 

site to request and utilisseo-reviews regularly, setting an 



important precedent by showirtg^that even well-respected authors/ 
need and accept -critique . This makes it easier for less ex- 



perienc^d authors to* request and accept criticism when their h 



0 / 



own 



lesson is reviewed, 



FUTURE 



Finally, a review may be useful iven if the lesson is never 
revised. Although we encourage revision, it is nevertheless" 
true that the ^reviewer may suggest a change, that is so nVjassive or 
difficult tha^ the author feels he can^iiot afford the ti^e to make 
the revision. However, when writing future lessons, the author 
can take. note of and observe the reviewer's suggestions. 



. \ 
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10 - WHM IS THE. RELATIONISHIP OF REVIEWING TO STUDENT TESTING? 

Is one better than the other? Does one come before the other? 

\ -\ •■ : ^ ; ■ • - ■ ." 

Whydo'both since they seem to have the ^ame purpose?- 

f' I ■ . ' ■ 

* It is a common misconception that student testing will ' . 



reveal ail of. the important shortcomings of a lesson. Thougti 
it -1® true that, in general, student testing is used far too 
litt^le and typically too^late, there exist several types of 
problems that even a perceptive author-using critical students will 
often overlook. Several of these types of problems are asso- 
ciated with objectives. For instance , students are seldom aware 
wheWthey are not being tested on all the 
objectives of the lesson. More subtlely," 
'^>^^'^^§^^}^thor s consistently^ tes'fi^^dhly with 
Recall items, when the objectives or 
the career performance requires a higher 

> ^, ^1.^ T • Military students may find it difficult 

level of thought — such as ^naJ:ysas^, ^ . i u \ li 

° ^ — " -to. comment negatively about a iiesson 

synthesis, or problem solving* Another ' ,r - 

problem is that since a student generally goes through a lesson 
only once, he is generally unaware of and thus- insensitive' to 
alternative branching schemes.- - ^ \^ 
Even vhen students can detect problems, they can fgenera^lly* 
only ihdiciate symptoms rather than diagnose the problem and recom-- 




•IV. - 



Vmend a cure. Fur theitnore, because of the rank and roles Implied 
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by the student-teacher re;L^tionship7 military students :(and per- 
haps students in industry)' may find it difficult to cogent neg- 

atiyely about a lesson. Since students generally go through k 

'. /■ ^. - • ' 

lesson only once, it is dif- 
ficult for -them to fee both 
learners and critics, simul- 
taneously. / 

While aware of thgse 
arguments, sojn.e:^people feel 




neyeiT^t-heless, thJat testing 



Some people feel that testing with enough ^students 
is sufficient. \ . 

with enough students is sufficient. In addition to^ ^kngers al- 
ready pointed out, there is av further problem. Running ^ large 
number of students as. a debugging technique requires rather ex- 

^tensive amounts of time devot)^ to aha'lysis.of stq,dent per- 
formance . Although a great deal can fee learned from workings 
through all the responses collected from ten to twenty students, 

. the same procedure applied to fifty or more students produces 
more boredom than information because of the large amount of 
redundancy. Thus, if all the^ previous ^arguments have failed to 

^.persuade the author of the .benefits of review, perhaps he can 
view it as an alternative/ to the decreasing return he gets from 
running more students. - / 



/ If one is convinced that reviews and student testing are 
/both necessary, there arises the question of which should be done 
first. Coding reviews should be done with Some definite purpose 
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in mind and that purpos'e dictates when the review should be 
done. Subject matter- reviews should normally be made just be- 
fore the first major student test,. Tn-progfess instructional ' 
desii^n reviews occur during development. Hence, the only order- 
ing; that is prone to much controversy' is when to make the end-of- 
lesson instructional design review. The end-of-lesson review 
•'sliauLi-in^Treirtr^^ the student's' time is very 

^expensive of if the pool of students for actual use is small and 
must remain '"uneontaminated. ".^ The latter situation sometimes 
arises when doihg educational psychology testing. In the above 
cases the reviewer^ serves in a sense as a |^.viine^""pig. " He 
smooths the rough ,edges .of the \l0asdn and conserves the precious 
resource, s^tudents, for final polisfiing. Again we.s^tress, 
however, that a reviewer is only a partial replacement for real 
students. . ^ 

' In nearly every other case, the end-of-lesson review should 
follow the first student trial. Though we of the MTC group haye 
never performed a review with student performance data at hand^ / 
we would'prefer having students try the lesson first: any 
obvious mistakes , embarrassing to the author, can be elimi- 
nated before "the reviewer sees tha lesson. / Also, after evalua- 
ting plerformance, the author has a basis fpr ^aiming" t1ie review. 
That 1^, he may ask the reviewer to solve a particular .problem. 

For the following reasons we feel the author would have a 
better psychological ^*set'* for accepting criticisms from the re- 
viewer "after , running a sizeable, number of students. 



• Either tbe author knows the llesson- has problems and 
appreciates the' reviewer 's help or he knows it is 
, fairly good and isn't crushed by the review. In the 
first case, the author shouldn't react negatively to 
the reviewer. In' the aecond" case, the author should 
more easily recognize the B«,view for what it ig, in- 
tended to be — suggested improvements to a viable 
lesson . : ■ i 



Student data can aid the, reviewer to find problem,s in 
the same way it can help the author. The reviewer ma> 
be more experienced at interpreting data and may find 
additional problems as well as teach the author how . 
to,/'read" student data. The reviewer's task of con- 
vincing an author tq make a change can of ten be sim- 
plified if the student data support? and justifies 
the reviewer's comments. 

Though a check on the data collection programming is 
^ part of a review, the check is best made by attempting 
to follow the progress of real students and by trying 
to evaluateN^ student only on the basis ,^pf the infor- 
mation which was put in the datafile. 




11 - WHO SHOULD REVIEW UESSONS? ' . " 

We stated earlier that the reviewer ^should be art outsidi^r 
with respect to the developj^ient of the le,3Sdn being reyiewed>and 



that -the reviewer need not have mastered the lesson mater^ial^ That 
st^ll leaves unanswered many questions' about selecting a re- 
viewer. Let's look first at the question of what art "outsider*' 

■ ■ 

is and ,Jphen examine the reviewer as a person (section 12). 

Tbliree categories spanning the range of closeness-aloofness 
seem useful.. At one extreme we have' the "professional" reviewer, 
in the middle there is a member o£ a course development group, 

and at the other extreme, a close colleague,' Experience, ob- 

' ' ft. • 

jectivity, perspective, and formality are traded off between 
the different "levels" of reviewers^' 

Aloof. * . Close 



• Professional : Group Colleague 

. ' Member ^ 

THE PR0FE5,^I0AiAL . . ^ 

The "pro" is somewhat"* akin to a journal editot. He i« les's 
likely to be biased by friendship and personality traits. In 
fact, he may be working with authors whdm he doesn't know. Fur- 
thermore, he is likely to see enough lessons to have an excel,- 
lent f,eel for what is good, what'is.not good, and precisely hpw 
to make impf ovements. As an expert, he is probably especially 



fast and efficient in reviewing. 

He faces problems liowever. Feedback from an author about 
how the review wa-s received ;and how useful it was is difficult to 
get. With anonymous reviewing, it is hard to clarify points of 
confusion or receive answers to redirected questions. Even with- 
out .anonymity, novice authors are unlikpl y fn np pniy rh^ H'^ng*^ rh ^ 
'^expert^s" opinion. In this role' there^ is a danger that the 
reviewer -would grow to be hypercritical^, as drama critics seem 
to many people today. Lacking feedback frdm the author, the 
reviewer is unable to see the effects of <^ his suggestions, Sug- 
gestions that ;are rejected or that fa.il Jto cure problems cannot 
be dejtectfed with such feedback. 

^If the professional reviewer and the author have met at 
least once face-tb-f ace, there seems to be an increased cha^nce 
for good rapport. The reviewer gets a chance at ^ quick size-up 
of the author and cari^* brief ly explore hi^ background, experience, 
and temperament. The friendship can he continued fairly easily" 
over the terminal once the initial face-to-face contact has ' 
been made. 

GROUP MEMBER '" ' ' ■ 

Compared to the 'j.pro," the member <)f a'courseware development 
group who is appointed '^reviewer*' has several advantages. Like 
the "pro,*' he has a clear , role and responsibility and will gain 
enough expertise to turn out a consistent, high quality pro- ' 
duct and enough.' experience to measure lessons by broadly-based 
standards. Because he knows the goals jxf ' the lessons and project. 



the target population, and the project backgi?'ound , he can be 
quite .efficient:. This familiarity means thatjr his comments can 
be miore specific and he can adjust them to fit the personalities 
of the authors. Hopefully he can strike a good balance between 
maintaining his own objectivity and bruising :he egos of the 
authors. "His comments may find easy Acceptance if the ' respon- 



sibility Eor a quality lesson is perceived by*members of the team 
as a shared responsibility. However., depending on His formal 
and informal authority within the group, he ijs not usually in a 
position. to enforce revision according to his review. . . 



THE COLLEAGUE 



The most typical sort of reviewing , that go'es on at present 
is done between a pair of clbse colleagues, often o.:f f icemates. 

* 0 

Such relationships, when bacJced by mutual trust and friendship, 
can help both participants to grow rapidly. By dping reviews 
of his colleague's lessons, each author can more easily accept 
a review of his own lesson. There is a commitment to implement 
or to "argue out" proposed Changes and, in general, feedjpack of * 
*all kinds is easy to ^et. To maximize feedback the two members 
should 'be"*5pproximately equal in position and experience. Though 
they. may npt shar^ final responsibility for the lessons being 
produced, the reciprocating nature of the involvement tends to 
keep the review careful and thorough. Another advantage is that, 
such reviews are often V'continuous" : the lesson Is reviewed at 
various stages of development (even before anything goes on-line) 
Because of this, less author ego is involved. Unfortunately,* few 




of these reviews are recorded formally and, hence, much of 
the effort may be wasted because of forgetfulness. 

The description of thejthree levels of • association between " 
reviewers and authors suggest that ther^^ is no clea^ answer 
as. to which is best ~ each arrangement has its own aidvantages. 
The best situation is a combination of ..all three , (All three 
types of review for every lesson would, of course, be too expen-t 

sive.) Authors should pair. up to review 
each other 's' lessotis during develop- 
ment. Each lesson should be examined 
by the group's reviewer in order to . 
maintain a consistent style and tone 

The importon\ chonges^ore reoMy mode throughout the group's lesSOns. Fi- 
to outhors, not to lessons, ' * , - 

nally, the grpup reviewer 6an send off 
especially good, bad,^or dif f icult-to-review lessons to the 
"pro,** In this way, the group reviewer grows prof essional^ly 
and acquires a broader view of lessotj quality. He also has 
a mechanism for finding aiid sharing new ideas as well as cali- 
brating his own work, , . 

Finally, the kinds of changes tTfat any type of reviewer is 
likely to be most successful in getting the author to make are 
the least important ones: cosmetic changes to display, tech- 
nical corrections, programming errors. Since our staff works 
mainly with inexperienced authors with little or no education 
background, our reviews are directed not only at specific lesson 
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problems, but at building a more long-range educational. per- 
spective. 

The important changes are really made to authors, not to 
lessons . Influencing authors to use more indiviudalizatfon 
and better questions is important. Introducing them to the 
concept^^ of mastery learning and taxonomies of learning is 
important. Lesson review can be an end in itself, but more 
importantly it is a means to expand an author's horizons. It 
is difficult to influence an author's philosophy by giving 
him a few typewritten pages. No sjlngle lesson review can cause 
these major changes. However, a series of reviews, coupled 
with student performance data, have caused significant changes 
in receptive authors. 

n. . ■ 
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12 - HOI/ DOES ONE SELECT A REVIEVCR? 



BACKGROUW . ■ . 

in terms of background, there is Tno obvious' curriculum 
or coursework that is necessary. Naturally, it is useful fo 

. have familiarity with the concepts and jargon associated with 
curriculum' development, educational psychology, and educational 
technology. All of the reviewers who have worked in the MTC 
group originally lacked background in some of the areas listed 
above, yet all picked upthe basic ^information; hence, don'f 

^ weigh formal training' too heavily. On the other hand, someone 
who is familiar with objectives, learning theory, and instruct 
tional design is^ likely to be able to find and justify prob- ; 
lens with high efficiency and clarity after , on J.y brief training 
Experience with so-called ^'packaged*' instructional media like 
programmed instruction, video casettes, and QBE is rare but 
* valuable. Persons who have tested such materials with students 
, bring additiotjal insight. j 

PERSOmLlTV ) . f 

I 

The personality of the Reviewer is probably the mq&t im- 
portant criterion. 'In a technologically rich medium like CBE 
it is frequently easier to find staff who are task oriented, 
Many courseware develapment; tasks can be superbly handled by 
the task oriented staff , but reviewing is not among them. 



^VAcceptable personality traitfe may>^ary with the role and adminis- 



trative structure, but s^v€r 



ral qualities are necessary. The 
reviewer should h^e^^iiumble,' don-perf ectionistic , and flexible. 
I CriticaL-^rgviews can.be very damaging or discouragdng to an 
r lauth.or. Careful phrasing of comments ^ can' minimize the shock 
of a review-'! Sensitivity tp the author's coifiments and feed- 
back can reassure^ the author of the overall worth of tiis les- 
.son. Because the reviewer Imay find himself in a position of 
oniy informal authority, rispect and credibility play an im- ^ 



portant part in his succes^ as a reviewer. These are earned 
by exhibiting competence in reiHewing, buts^re also a facet of 
one's personality. 



TRAIWING 



'Tjext to personality, training is probably the most im- 
.portant aspect of a reviewer's success, ^.^s stated earlier, 
reviewers are botji "born^' and "madey'* The emphasis seems to be 
on the "made'l in our experience. Knowledge gained by'ffeviewing 
lessons grows at a terrific rate. By the time he has completed ' 
his third review, a staff member either has dl^d^ reviewing 
isn't for him or has laarned so much he needs only minimal 
-guidance in doing future reviews. 

Three, techniques allow the novice reviewer, to quickly * 
acquire the tools of tihe trade. Examining reviews done by other 
reviewers gives him pointers about typical lesson problems and, 
more importantly, thej style and tone of a review. Secondly, an 
appreriticeshi]) within experienced reviewer is extremely desirable. 
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The experienced reviewi?r can examine the written work of the 
novice and prevent him from saydngj some thing that the author 
might find too critical or might \mi53interpret. Thirdly, feed- 
-back from the author about the review .is necessary to refine , 
one's techniques ho matter how ^experienced the reviewer is. 
If p6ssible, further in.fbrit^tion may be gained by examining the 
lesson a fe'w^days ahd weeks a^^^Ts^the^ review. By examining the 
"last edited'' data for each block and by^'^we^^log through in 
student mode, the reviewer can determine which comments were ' " * / 

convincing and important enough to cause jihe lesson to be 
chang^. 




STAmAWS ' 

The guidelines in chapter II were developed to assure that 

a • ■ >. • a 

several important considerations and comparisons were made by 
all reviewers as well as to assure tl^fe delivery of a product con- 
forming to minimal standards irrespective of the person perform- 
ing the review. A high level of uniformity seems difficult to 
achieve, however. Different reviewers have different sensi- 
tivities, l^at is achievable are standards that specify types ^ 
'^--^a^^ levels of e^^amination which' are detailed enough to produce 
useful feea%SLck to an author. Giver* the admitted situation of 



non-uniform reviews, one is tempted *to request revie^s^ of a 

V. ... 

lesson from several reviewers. The main disadvantage to this 
is economic: the thorough end-of-lesson review cfan'take from 
15-25 hours (including wr it^.-up) ; in-progress reviews can take 
from 3-6 hours. .5 
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VJHAT ROUErStSPONSIBILITY> AND AUTHORITY SHOULD A REVIEWER HAVE? 

This quest ion .arises most typlojally vihBn^,^^^^ %XeT 
viewer Who^J^s'a member of a curriculum development group. In 

i:^ position, the reviewer may assume a role either as an - 
assistant ^'to the project director or a member of the ^uthor sup- 
port team (which might al'^o ihclude" artists, photographers, 
editorsj etc.)."'. Tn the former role the reviewer ^s powerful^:.. 
but he can'become too evaluative. Further, his goals may be at 
cross purposes with those of other administrators (e.g. , high 

quality versus high production). In an author support role he 

■- ■■ ■ • ■ ■ " 

is easier for the authors to ignore, but is "on the author's 

side." 

A second yd imension of the reviewer's role is, whether he \ 

* / 
\sb6uld also be an author. A full-time reviewer may be seen by 

aut^hhrs as living in an Ivory tower — unacquainted .with th^' 

real \world. On the ^other„ hand , extensive , 'ongoing lesson/writing 

^ \ ■ . . ' . ^ 

experie\ice^is not a co-requisite for good reviewing; our best 

.> ' ■ ^- . . ' 

reviewers h^y^ written few PLATO lessons. / 

. - . . y " ' ■'.•/..'■ 

No matter what the *role, it is very important ''to give the 
reviewer responsibility for the lessons he reviews. The res- 
ponsibility may b4 joint, that is, shared with the author, or 
advisory, that is , the reviewer's responsibility ends when hi^ 
coiiiraetits are .delivered and documented, • 

Na^discussion of • responsibility's complete without 
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-^iscussing authority. Formal authority seems unimportant in re- 
viewing: excercising it tends to lead to evaluative reviews. We- 
op^T^ate'T*^^^ necessity, strictly by informal authority. In our 
case, all requests for reviews are yoluntary^ and we have abso-^ 
lutely no power to ensure that the reviews are read or used, 
in this situatiofi, one must maintain a reputation for high 
quality work. Since our -Service is advisory only, it's easy ^ 
for an aijthor to stop asking for advice if a review is unsatis- 
factory — - as we have found. Necessarily , feedback from the 
author to the reviewer is critical in this situation. A 
phenomenon we have noticed might be called the "it's the last 
play of the game they talk abput" syndrome. In.the^same way that 
basketball fans curse the player who missed the game-winning shot 
(even though he made ;30 points during the gameyT^aiT author tends 
to best remember the last review ,' — once ''turned off" to reviews, j 

he is unlikely to request another. . / t 



VHAT CAN GO mONG WITH A REVIEW? 



CCVm REVIEWS 



Lesson reviews cjn be dangerously mislead ii/g or ambiguous 
if the reviewer- is noi experienced or is in sonie way unqualified. 
. People doing code reviews, for example, tend Wo get carried away 
with* saving vaWabl^s, words, .units , or whatever. Optimization 
at the bit /level is not generally applicable^ to PLATO lessons. 
^_,J[lJLg^£dSi:seoding using esoteric commands ' whicl^ the author cannot 
understanjd produces no long-term gain3. |temember: the goal 



• is to q4i 



(^hange authors, not lessons. 



SUBJECT MATTER REVIEWS 



Subject matter reviews are al$o /subject to misdirection. 
Because- many subject matter experts are not trained, or exper- 
ienced as reviewers, they often ignore lesson objectives and 
student backgrounds. The resulting suggestions, if implemented, 
may actually degrade the lesson. For example , some subject ^ 
mattr^^r experts ruthlessly root out all half-truths. In the intro 
ductory courses of tffe physical and social sciences many prin- 
ciples an^ rules are giv^n without starting all the exceptixiris^^ 
Though it is unlikely that the studenj.^j^«^ be conf rpnted^^^ 

with a liquid thaX^j^fen^*'£rTreeze or a mammal that lays eggs, 
such things exist. Many subject matter reviewers point out these 



exceptions with too great a* delight. This forces the author to 
complicate each rule with qualifications or to list the excep^^ 
tions. The lesson objectives might have been to teach broad '' ' 
principles and generalities, but tjj^-^thor ends up adding use- 
less information just to,,,,e^^isf y the subject matter expert/ The 
student, feels that if -it's there, he should learn it. 

One way to discourage the practice of overqualif ying each state- 
ment, is to follow this madness to its logical conclusion; if 
it's worth Wntioning, it's worth testing. If the material meets 
this criterion it's probalily worth keeping — and.it should be 
ih the objectives. 



lUSTmCTlOHkl VESIGN REVIEWS ^ \ 

' ' " ' j ■ . _ ' ■ I ^ 

Last, but not least, is the inst^uditional reviewer and his 
set of faults. In addition to just hoyiestly misunderstanding the 
intent of the lesson, the reviewer may have too high standards 
±deFlni^ the author 's abilities/, time constraints, etc. If 
the ireviewer does not adjust his comments, accordingly, he ean 
frustrate the author. A fuller ^iscussion o£, problems for inr^ 
struct ional design reviews is ^und in chapter II, "How to Re- 
view a Lesson." 



CAN A CHECKLIST BE USED TO^REVIEV^ LESSONS? . 

We use a mental "checklist," but it's probably not what 
you would expect.' Our "checklist" is a condensed versi9n of 
the reviewing categories, riot a list of items or specific 
problems. This '^ay seem strange; many people watit specific 
items listed (e.g.. Is the/ -BACK- key -frequently active?), 

\ In fact, a number of so^c4lled "reviewing forms" have been de- 
vised. We don't feel they are adequate, however . 

Items on a list are typically 
ambiguous check-off's /or ratings. 
In the -BACK- key example cited above, 
the reviewer might be expected 

'/ to check "yes" or "no^l or to rate 
the" frequency on a scale ranging 
from "all the time" to.:"never." 
This sort of feedback to the author^ 

is unsupported, non-specific, and hence nearly useless. What 
the author needs to be told is, "Because the directions for 
the second drill are somewhat complex, the student may wish 
to press -BACK- before t)6ginning'^ the drill." 

. To be complete, a list of items against which to check 
the lesson would, have to be very long and most items would be 
inapplicable to a given lesson. A list of frequently occur- 
; ring "problems is useful ^for the author checking his own lesson 





A problem cannot be ignqrod merely 
because it happens infrequently. 
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arid for training new reviewers, but it is not .sufficient fair"" 
performing a review* A problem cannot be ignored merely 
because it happens infrequently. In general, .we have found 
that because of the diversity of lesson types, item checklists 
cannot provide a complete review. A list of categories which' 
we found useful is located on page 115. 



' ' Chapter IV - End-Of-Lesson Reviews 

' ' This chapter desi^ribes some special techniques used when, perfoi^ming 
end-of --lesson re.view. It includes general comments^ flow charts, and 
annotated printouts* from reviews we have *done . 



ADDITIOriAL TECHNIQUES 



Tn addition to following the gu,idelines of chapter II, 
thoro are some additional techniques which are especially use- 
ful when performing an end-of-les^n review.' The rationale for__ 
these spei?ial techniques-Is based on.' (1) the 'kompleted'"^ status . 
of the les'^o'h and (2) the..f easihilitv of inclu.^iing an annotated 
printout becaust\of a reduced need for imnediate (hours or 

days) feedback. \ 

• ' . • ■ \. 

\'Jhile writing an\nd-of -lesson review using a printout, the 

reviewer:- would add the following steeps to the procedures outlined 

in section 4. The follojwing steps would be done during the 

third pass through the lesson>v N 

Follow the. flow of the riR3son on the printout as you 
go" through the lesson. Take- note of any parts of the lesson 
th^t are on the printout bu t a re never executed. Make 
notes directly on the printout - Specif ic' comments on each 
part of* the lesson should be marked on the printout at the 
exact location of the code for that p^rtr_ In this way the 
author can easily implement those changes he agrees ^ij^-^h • 
Tn inserting comments oj:i the printout we generally color 
code the comments with three colors of ink. For example,, 
one might use rdd for misspellings "and puncl*tuation cor- 
rections, blue^for comments about the wording and fo^r making 
■g ene ra 1 suggestions for imprpyement ja-and green for coinments 
on code revisions that would be benefitrial* The use of 
colors^ gives the author or a later revisor an indexing * 
system whereby reference darf'be made to a specific type of 
> comment,. Two cautions: (a) black ink is difficult to 
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differentiate from the isrintout, ink and ^(b> a'tait 0p 
pen produces so much color that a printout may segm'to 
have "bled to death/* ^ " 
Make notes on problems that seem to recur through- ' 
out the lesson w For these criticisms the reviewer sho 
note the line numbers, of t^e printout wheire ^they^ ^€ur so 
that they may be easily referenced in the geniefal comments 

section of the reviiew. ^ 

^ ^. . 




- Level '4 corrections can be easily made on a printout. The 
effort needed to "point out a misspelling on a printout is minor; 
the psychological impact to the author is also minor.— Wtefi' 
reviewing lessons without a printou^, it is often not worth- 
while to point out spelling and punctuation" problems bedause 
the author may feel the reviewer is looking' too hard for^ things 
to criticise. 

Though one measure of author quality is the ability to- 
accept .constructive ^criticism rationally, we have never encoun- 
tered ah author who didn't react somewhat defensively to a 
lesson, review. What has been surprising is ,the intensity of the 
feelings. As an example of the intensity, one projett used to 
review, lessons with three people plus th^ author present for a 
final discussion/presentation. The pressure on the lesson's 
author was so great that the group decided to exclude the author 
froin the proceedings. ; 

A partial solution to the problem of intense author reac- 
tion is realized if you, as the reviewer, take a careful, ^cal-' 
culated approach to the construction of the review. Cooking ^ 
carefully and diligently 'for many good things to say about the 
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lesson is useful 'both as positive -re inforGepient and. as 
''soft^j)acJU.ti)^-^ '*hard v^^ord^5*". Remember: things that 

are- (ione well are luH so obvious as thrn^ls done, poorly. The 
effort that ^oes into revising a lesson segment four times will 
not fteneralTv be obvious to you, ' Clever coding may escape you. 
The author will probably expect the positive"^ comments you make, 
but will be surprised by the unfavorable remarks. To reinforce 
the author's sound practice, it's necessarv to make more than 
supbrflcia}. positive compliments. justification of your comments 
provides credence that you are sincere (e.g., ''That display cer-^ 
tainfv clarifies what must be two of ten. canfused concepts."). 

tone, of course^ is important. Suggestions given as ques- 
tions ('"Would it be >etter if...?) are easier to , accept .than 
orders ("Change this to read..."). In general, allowing the 
author an excuse rather than forcing him to admit a mistake is 
a good way to reduce the threatening nature of a review. For 
example, the reviev^er might use 'phrases ^ike, "It seems lil:e ' 
this technique would work here, but,,." or "Do you really want 
the student to,..?" None of these suggektions-for' softening 
criticism is particularly clever; they're just basic human re- 
lations, but -rthey aire imp'ortant. The human, personal • touch is 

also ^.mportant. If the revie^wer and author can't meet face-tb- \ 

• ■ ■ .• , ■ \ 

face, empathy can be conveyed by a cover letter or introduction. 
For this reason several sample letters ^are included with the. 
sample reviews. 



SAMPLE REVIEWS 



In the sample reviews fnund on the following pages, the 
ginal review is shown in the standard typeface and explana- 
tions and notes about the reviewer's comments are shown in 
script. These examples are based on reviews we have madq. for 
the PLATO authors at Aberdeen Proving Grounds, Maryland . We 

0 

must hasten to point out that these reviews have been loaded 
with mor^ criticisfhs than Were actually the case. 



CO/MPUTCR-BhSeD 

eoucwion 

. L/lBORKrORY • 

vtRS^rv OF ILUNOIS AT IJRBANA C HAMPAIGN- 




D, L. Bitzer^ Dir^ cfor 



Urbcinfi, Illinois 61801 



(2.17) 333 6210 



Sample 
Review I 



T<): ' Ron Richardson 

Fi\uM: .Mary Grahain 

RE: Revifc\w |)f lesson "3?atb 

DATE: June 27^* 197- 



Sorry to hive taken 's6 long getting this material to you, but I didri^t 
return from the site visit until the middle of last week. 

I'm afraid I have quite a lot to say about this lesson. So I thought I'd 
Warn vou in advance not to x^;orry too. much about it. The inain prbblems 
in the lessOn are about the same as those made by most, new authors — and 
old* authors, for that matter. ' ' 

I hope my notes are somewhat helpful. Please let me know if you disagree 
vvith or can't follow any of these comments. We can talk on-line if 
you'd like. . . • 



4 

Sample 
Review I. 



General Comments* . ' . 

' f . • , ■ ^ ■ • ■ ' • ■ 

^ e \My comments are going to fall generally into three categories: 

' ' . ■ *^ ■ ■,/•'•• ^ ' ' ^ . 

.1. An attempt to summarize my. impression of the content of 

/ ■• , - • - " . ■ . • 

this lesson 

2. An attempt to outline the precise skills that are rele- 
vant to this content 

3. A review of . the actual iipplementation/presentation of the. 
lesson ' " . 

Specific, comments about the lesson appear right on the -printout of the 
lesson. ' ' - 



Part I 



In this section I am briefly summarizing the main teaching point 
Of lesson "lathe/* The outline is incomplete, but I think it includes 
most of the important information presented. A checkmark next to a 
term or characteristic listed meant that you have included a question 
on that point. - 

I. Introdtiction ^ 

tool bitv A lathe ^'Cutting tool 

A wedge forced into the material — 
. ' ♦ , Results in a '*fupt{ire*' or ^^plastic 

^ " flow'* of the material 



n^dvimoA madzJ It ^ impoAtant that, tkd Advlm be iAJOJ^l-o^gavUzzd, condyUz 
and anaphlcaZJiy pZza^ant a.^ tkd n^zvlmoA JUi to fae .CAzdibtz to thz author 
04 an zxpeAt. HijpocJujiij corner too zo^aJLij. '^^'^—--^^ 
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Tool Bits -.Typos of'Materirl 



tool bits . 

carbon steel ^tool bit 



, Jiqpt are made of steel 

vl'lade of high carbon steel 

Economical ^ 

/Loses temper at relatively low 
heats - 



speech steel 



stellite 



Alloyed. . . ... 

Remains hard even at very tiigh 
V temperatures 

v^lost conmonly used for general 

; - machine work 

' Like carbon steel tool bits, .it's 
^ forged ' 

Wade of clfrpmium, cobalt aiid tung- 
sten ^ 

. ' ■ • • ■ '^^v ■ ■ \ 

Is cast, not forged 

Vused .mainly foi* Abrasive materials 
' ;iike cast iron of bronze 

{¥o.r biQvitij, tilt ^ut q{) tht oiJUtliniiXii^ daZ<lt^d.]'^ 

Tn addition to tha TTvat:erial listed, above, several undefined terms 

rippear in the lesson. I assume the student already knows enough., about 

them to proceed with the ^new material,. 

^ ^ Again, the list is incomplete but it may give you an idea of the 



\ ' • *The,\ aiitCU.n'Q.HiJCLb made liy Hit n^wi^m^ as^ pojit o^^ hoA amlipl^. 
It U p^iz6pvttd /le^e to 6hot^ tht atttko^ hoio the, It^^on tooi pokcoAvtd.^ 
pne. pU'b£e)Vi njXAj^td 6ubttij cut tliAyS poivit a lack chtck moAi^ be- 
icrfc mc^t tcpic6 pol^it6 out. ^omztlting tht author my have, bttvi amiooAt' o{^i 
a great diZixl oi in()om'citlon Is pie^brnttd but o\xlij a mafJi bit it iJ^ 
tt^tod. ' ' ' - / > \ . 
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type ot* thinft"! mean;* < 

The student is expected: ^ • 

• , ' / \ • ■ - . . ■ ' ^ 

/l.. To recognize the different parts of a lathe cutting tool — 

• a . the holder 

b. "the. bit • • > . 

c. the cutting ^dge 

d. the t6ol ^ 



To recognize names for different parts of the work .pl;ece — 



a . the.^shoulder 

b . the end 

c. the face ' 



H. the external part 

e. the internal part 

f. the axis 



To recognize t)^e effects on a work piece of the following 
operations — / - . c 



a . turning 
b* . facing ; 
c. boring 



v.. 



d. knurling. 
,,.e. ! cu1:;,ting threads. ^ 
f . making grooves 



Cs I«see it, these are the skills relevant to the content. of your 
^ssbn "lathe/' ^ I 

1. Given, characteristic, provide or recognize term 



2, 
3, 



^ven term, provide^^ or recdgniz-^ |:hara«cteristic 
Given picture, provide or, recogni|:e name of ^ item 
Given name. of item, select picturjs of it 



5. Dis^lmii^atie arfiong terms/picture^ presented by matching 
thiem; — called multiple discrimirtation 



• and aonazpt6 thz ^txidznt should be ^amlUoJi wMh oohun kz taku tkz luJ!son. 
I ShzJJ^jU^thz assumptions in a non-n.zp^oackiu^ my. HomvoA, by.6Z(Ung 
\ tKU LUt, tkz aixthon, u n^mindzd zxpZlcutty 'pi ^qvqajoI dzct.UloYU> kz has 
\ madz AjnpVicJXly . \ , 



66 



Sample 

Review I 



ThJ following matrix shows the relationship among these skills and 
the n.ilin subdivisions oi the lesson. .Checkmarks indicate some questions 



in tfxe. area - * 



Tool Bits — 
Types of Material 

Tool Bits Tasks 

Tool Bit Holders" 



1 ^ 2 
/ / 



/ 



As you can see, the lesson doesn^'t give the studept any practice at 

^ ' \ . 

atl'^in some of these skills. And there is not much c^iance to practii 

^ . i . 

s^one of those cheeked.'*^ , ''I 

. ■ . \ • . 

\ 

•One th'ing you can do now j-S go back to your (or my\) content outline 



of the lesson /and decidf. what you really want the student to knou^. Then 
dump out whait^ver is left over.*** . ^ \ 

If. "you wa/nt the student to know the important characteristics of 
tool bits mk holders, you mJist 'give him an opportunity to respond to 
them. If ndt, you can eliminate the extra details and save him time." 

Next, ir-you look at the skills I' v^ listed, you'll see that the most 



noi^, tkz autlwn. should be picking up tliz patWin and be ^omoidfiat 

''{at idiat to comn. ' / 

■ . ■ / 

**PincilZy, Iwxz comu thz me6>6age. T/ie H^zvimdn. h(U ''toM^^gnRpifidd .. 
hz% panckoJk'' 6o tliat tht amthon. >i^hoatdn^t be ^uApnJj>td. Th^ zy/dzncz i^ 
M thQAZ. ^ . / 

***HcAe ^4 thd ^solwtlon ioK tiiz aivtiioK^ W^en the K^vlmoA dou 
-iuc/iTt tho/iougiyjob dmoki^tMtiiii^a pn^oblm, it' 6 'cmty {^ciin, to 6tioi<)'tli(i 
atitlion a way oi/t: ^ . * - • 
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complex one (5) is .not dealt with by the lesson. It is important )Lo 

'-^ 

include. questions that help the student tie together "all the individjial 
facts youWe given him, )tn that way he will get to understand the rela- 
tionship among them. 

Finally, asking one question per fact is just not enough. Even if^ 



th 



e. point is go simple that one .qq^estion should^jjod^-ea'Ceiinderstanding, 



\ 



it is important to "seed in" questions on the ^^saiigu^^olC^ at intervals 
throughout the lesson in orde,r to reinforce the student's grasp of the in- 



f orma tion . 



/■ 

/ 



A useful way to accomplish these last two at the same time is to 
review items simultaneously — that way you allow the student to tie them 
together and get more practice all in one shot. 
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Dear Fred , . . '\ 

Mv coninents are divided into three major sections simila^ I^S^'Mihe' structure 
of pv review. of "drills":* ' ' ^ 

1. ^ ileneral comments • 

2, Ftow chatts and" related comments ' _ . 
3i Jipecific coimieri4^ as written directly on a printout of 

"keysJ^.^^^-.-fPTease note that these are "coIof-coHed as^: per 
^^he^^ey on the nrintout) . 

This rex/iew is obviously f 01:^ your benefit and therefore you, should be 
the* person to decide on which parts of my comments are applicable,, war- 
ranting changes to the lessen, and which parts are not.** I only hope 
that you will "review" — s^ that L will know how to direct my future, re- 
views more precisely to your needs.*** After you have had a chance to 
analyze- these notes, I will check with you on-line and arrange for some 
Cirrte x^fhen we may get together on the telephone to work out any specific. 
idj>as you may have. ' ■ 

Thank you very much for your time 'and patience. If I or my organiza- 
tion can give you any further assistance in either programming or edu- 
cational development, please donVt hesitate to contact me. 

Sincerely, - . ' . . 



John Kiley 



[a commit, tko, x^vlQ,m% 'mak<i^ tatoA In tku ^tvlm] ha i{ee£4 thz author. 
me,d6 a map oj$ ttit Xiwlm. . " ■ , , 

- **T/tt6 60^ oi •t^zn.tzncz Included 60 that tha autlioK knom tiiat . 

•f/ie .icucetoe't lb not tAyytng to gmb authonAt,ij tlvtt iMn^t hiM. 

***Feedbatfe to xavim^a/i ^ mpoxtant. A 6t/iong n^zqaz^t iox tlvU 
' ^(iQ.dback cdJbo bliom gtnuA^nit conc(ixn\ioK^th^ autlioK^6 IddtinQ^ and ^(i(iction6 , 
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General Comments • 

^^^^ 

A.. Organization 

In flow chart 11, 1 have outlined an alternate way to de~ 
liver the material in "keys.'* The major changes from the present 
version are described below. Also included is the reasoning be- 
hind my suggestions. . . 
1. Pretest 

Your pretest is an excellent idea. However, shouldn't 
a student who takes the test without studying the lesson be 
required to pass the same final exam as hi^ counterpart who 
went through the entire lesson?* That way you are assured 
that all the students who complete *'keys'' will have the ^ati/e 
minimum level of mastery. 
"/"^.^--End^-o Test'' , ' 

I think^iT^iplr-W-jir^rxJ^ the questions ./In 

each section con^tained only drill questions ^nd^TF'^tTie' 
had one final test to evaluate the student's knowledg/. That 
test could be organized so that if the students received more 
than a pre-specif ied number of ' wrong answers in a ipe- • 
Gific area, he would be sent back to that point In the lesson 
where the information he lacked was firs^t cover4d. Then, 



*The. authofL had a tino-oji lo^i^on vklth &tud&nU' aould'complzte. ioK 
JuU cJizdit. Att2AnativeIy, the.y could take, a pfieXoAt .and, iuccfcA^- ■ ' 
^ul, bifp<u6 all iMtAuctlon. Jn ,tiiQ. e.ve.nt tliat tliaj took iliz iMtfiuc- - ■ 
tlonal path thAoug': the. lQAi>on, theAe mu not an zy}d-oi-lulon tut. 



ERIC 



70 



73 



L 



Sample 
Review II 




after complet/nK that sec.tion, he could be returned to the 
area of the/test in which he had been working. That is repre- 
sented in/flow chart II by the red lines going from the block 
"End-of/Lesson Test'* to each specific subject area, 
Order/of Representation 

On flow chart II you can see that I have placed the 
'c/eneral Data'* first and then followed that with specific data 
bn each of the keys. Because the "General Data" applies to 
every key covered in the lesson, .1 think it- would be helpf ut 
if the students were introduced to it first, so that they cat^ 
apply it to each different key as it is encountered in the 
iesspn. . 
*'Help Sequences \ 

..a. I have included in flow chart II five help sequences, 

one for each major key and one for the general data, sec- 
tion. They woulN^ work similar to those you designed for 
"drills'*,* In ea^ subject area the student's errok count 



\ 



would be initialize^ io zero. During^THe^^±i±^,~4f--t:-hat:_ 

\/ - ■ • \ 

■ i ' ' \ 

count went over a pr'eset amount (in flow chart II, I su^- 
gested '*error=3" an'd have depicted that branching with 
a red line), he v7ould automatically be put into a help- 
type sequence where he would be given remedial instruction 



*T{ie AzvieijpQA ^f^f^dcJuvoly raying , ''I knou) ijou oKa tmuU enoagfi 
to tiiink Of5 thMSp hdcojUd yoa^va dona it baioK^L., faniheJmoH.t, it mi a 



good idaxL." 
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\ 



and then returned (re-setting his error count to 
zero) to the place he had been in the lesson. The lielp 
sequence would v^lso be available via the -HELP- keyl 
That way the student would never get ''hung u^"* at an 
arrow- * - 

On the other hand, you don't want to bore th^ good 
student with too much drill.** To prevent this, yoU 
could have PLATO send the student onward to the next topic 
(this is shown on/fche flow chart as a green path) after 



some preset number of consecutive correst responses. (For 
example, I chose ''correct=3" in the flow chart). This 
will keep the student from becoming bored with material 
he already understands and also" allov; the good student to 
cpmplete the lesson in the shortest possible time.*** 

Nate: A way to write ^ the code to do this branching 
was suggested by fJohn Smithl:ias follows: 
' [Otlttzd iion. bKzvltvi] 
The other form of help sequence seen on flow chart 
TT li^~^TOTe-that ^he^t^ at each "summary'* 

block. Those paths are depicted with lavender dotted lines 
coming from each of the two summary sections. After 



*T/ie KQvlmoA IncUA^ctly raying that a 6tad^nt could gut txappzd 
by a qu^tlon hz can^t anMonn.. HomvoA, Ivustoxid dioMXyig on. thz piob- 
Im, hd po^int^ oat a po66tbtc 6oltxtion. • 

_ ^ **In ^eie 6ci^itcntu; the kzvIqwqa \a handinq out 6otm tiUtnuctionaZ 
m^g n pkUo6ophy ~- — and xi my to Amplmznt^i^ 



. ***A le^6on c\fiaA,a(it(inJj,tlc to onz oi tkz pKojzcVh ovm- 

aZZ goaZ6 — ^t£me 6avtng\. 

11 \ 
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studying the information in the summaries, the student 
could elect to see any part of the lesson material for 
which he feels the need for further review. .That sort of 
branching could be programmed without requiring any new 
text insertion in any of the major blocks, 

■t ■ ' ' ' * 

5. "Full Summary" Block ^ ' 

I suggested- adding this block because I thought the stu- 

dents would^ be helped by seeing all the data together in one 

place and because it would allow them the opportunity to 

branch back into any part of the lesson that they may want to 

1 

review before taking the final test (see part 4 above), pne- 
way this summary may be set up would be in chart fotra. For 
instance:* . 

Key Physical Use Tools to Techniques to 

C har a c t e r i sties (Pro &.Con) cut Keyway cut Key way 

square • 
round . . ♦ ? 

B. ''Lesson Objective^' 
— — - ^Lj'iie^^r^ent lesson objectives (lines 594-609 on the print- 
out) seem like they might be redundant,** Most of theTiiTolrm^rtioTi — 



*Sp^cA.^la dojtaJMi of^^hot^) 4agg.e4i^0Ki6 might be Ajnptmdvitdd 4em 
Qm^AiJi^y mlcorm. Though tli<i dxaoX mplm^yitcution may not iolZow tm 
4aggc^^on, mvtoJiij ^aggt^ting a ^^choAt lonm ^ummoAy^^ 6up/ . 
pf^e6 too' iJjtttd ddtcUX.. ' / . / . 

**Rei5eAence4 to pHAMtout cuid dom u;/ie/te po/^^lblz. / 
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in the second paragraph is contained (and is well-st^'ted) in 
the first.- It. seems like ypu could easily condense /those two p^a- 



graphs 'into one concise statement . 
"Introduction".? ^ 



/ 



The "Introduction" section is, one of the mo^t important sec- 

• ^ ' . ~ :-/ . - 

tions in the~ lesson, I see it serving three main functions;/ 

■ - ■ ^ " 1 

1. Carcefully indoctrinating the student into the subject material 

o ■ - ■ / j 

of the lesson - / 



Supplying the student 
is to follows In this 



fjith a ''road map" of- that lesson which 
.way, the student c^n get a feeling for 



where he is in the lesson and what directions^ he could pos- 

/ ' ^ • 

sibly take / ' ■ ^ 

3. Telling the student aboiat how the "help" sequences work and 

-' i ... 

how he himself can fx;nd the information in the lesson ^- 

without the' aid of a proctor 

Your introduction does an excellent job on "1" and "3" but is 

rather brief on "2."* Would a table of contents help? 

D. Titles on Frames 

I. When you introduce a new topic on a new frame it is usually 

c a gQod idea to place at the top of that frame an appropriate 

title (in relatively large letters — maybe size 2). That 

way the- student sees at a glance jiiHtr what t^p4c^tJie .frame 



iohat tuai dond wdil hopt^uily mcouAagd thd author to coyitinud to 
mpplij a ccvidivJUii] con^tmictdd Int/ioductton. \ 
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will cover and realizes that the past topic has been completed. 
(This is very similar to chapter and section headings in most 
bcy^ks, and serves a very necessary purpose. 

In rVC^ren#^. to your -lesson as written, I have put remarks on 
your p\intbut suggesting a title.^4t the introduction of the 
following units:* ' . 

unit name line location on printout 



squ 

squ6 

squ7 

squ9 

squlO 

rdl 

rdsum 

wood 

line 



246 
396 
435 
485 
518- 
666 
715 
745' 
853 



II. Flow Charts I and II - Related Comments 

On chart I, I have tried to depict the structure of "keys" as 
I see it. It seems very linear (i.e., one single path with little 
branching allowed) and gives the student only a few chances for re- 
view. The green lines indicate normal progress through the lesson; 
while the red lines indicate a PLATO- initiated branch for remedial 
instruction. 

On chart II7^^ye tried to depict a restructuring of "keys" 



i6 no Aojcuon {^oH, i/ie autkon. to dwpticAta th^ ukfKk oi tkz 
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that I. think would adapt to a broader range of students. • The 
key features of this are explained in part A of "fhe^enefal 
Comments section of this report. Again, the normal student pro- 
gression is shorn by the green line. PLATO--f greed branching (in- 
cluding remedial help) is shown in red. iStudent-initiated re- 
view sequences are shown in- lavender.* \ S 



if5 



couMZr-^^ ci^ tliiL6g colou can 6c 6hom in tht ioltoiolng 
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FJNAL 
STATEMENT 



jumpout 



INTRO 



I LESSON OBJ,. I 




■■ WORK ON 
SQUARE KEY 



..GENERAL DATA 

1. CHORDAL SEGMENT 
0.) DEFINITION 

b. ) HOW TO CALCULATE {FrORMULA) 

c. ) USE OF TABL^ 

2. MEASUREMENT OF DEPTH JO^^ CUT 
USING SQUARE KEY 



TABLE 

FACTORS ON 
CHORDAL SEG. 



FLAT KEY 

1. CHARACTERISTICS 

2. DIFFERENCES FROM SQ. KEY 

3. USE 



ROUND KEY 

•1. USE 

2, DESCRIPTION 

3, MACHINES USED TO MAKE KEYWAY 





SUMMARY^OF DATA 


1. 


SQUARE KEY 




2, 


FLAT KEY 




3. 


ROUND KEY 





WOODRUFF 
1. TOOL USED TO MAKE KEYWAY 
0.) HOW TO READ CODE ON TOOL 
bj DRILL : READING THE CODE 
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cn3H £=JJ« 



9 

CO S m 



I 



d13H C-JJ» 



II 

5°= 



en3H 



£=4J8 
I 



1 1 
1^^ 



is. Us 
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I 



dl3H £=JJe^ 



II" 

E 3 



< o 



• e i g i 5 
^ o s £ i ^ 

5 ?43S ? 8 



III 
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« 
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'^9 *jmt 

150 term 

151 write 

o 

153 arr,o>g 

■•154 , 5p€CS • 

.155 putv 

156 bumf>'''"' 

157 ittrcng 

158 jMci^e-^ 
15'? an*// 

161 * rtex t not^ • f nti' =.h 

162 no 



iat, 1310>Hhat is thi^ diameter of a *807 Woodruff 



s 



"7^ 



J" 



I 

/ * 



1 '^riOT^e . 

(-.ov*~n ei -;bts,7 ^u) < " , inches > '-tj- 

/ 



aoniAjdoAed pancJmtlcfn bg 
PLATO and hence. a*e ignored t 
Voa p/Lobably Mint €o acttpt 



164 unit a""' 

1.6 ..rif. at vour handout on the ^^^^J^f^^ ^v^^^ ^ynonyttts uz., i(t.vzn^tiiMk& .V 

lx*7 ' , t .iti'* -, an/:J^ del: t-'T III in«-u'h'^t .distance a *404 w^^drutf ^ ' .-^ 

IbB iJ'-V 3houi'^ proi i ude ab:ve the ehaft'^' 

170 iv^-ong. Q 

17! 1 udge ior e 

173 vr it:*.! Fv-^l fi?rTe, you s^'T'm to kn^'.^ you e>t^uff ^bou^ key^r^ 

• ' .ri>"jd }'T:"/'>''<r*V3. ' ' ' ' . " / *' 



U 173 



l^r? 



i'iKrrh voM .rtre l^vrKm?^ fr*'^ 13 the C 



read 



• / 

irt^ori t he 



7...-...: 



175 J^uit 

i8i -^t \. 

1??"^ inj ite 

1;?4 5i7:e 

187 
1^8 

la-?^ 

191 dv;^ ' 

19 * t .iii ^.«lr 

(Jl-^ ; ?l 
. 194 lie 
19^ 

19^3 r.iu:-.'? 
19'3 ' 



/ 



- par t « 1 , b i.o*: M -d 



/ 



"SSON OB JECT I V|s 



I. ^ 

,,,,3 ^ : — — , — ^' ' ' 

mu^ be abl<! to proper 1 y , fnd*?nt i fy tVie • 
C' -mm^iily us'?d k<=vi;. ancl k^r^'ways, dnd calculate t^ie 

i n '^r-ti i. f or rr»^?': h 1 n i n^e '^^ he ke'.vn*'.?y» t o t V le con e^rt 
c^m'-ri3ict?3 to 'AM f hin ;^,#(ri; , iirar^: ihe procedures as^ 
r t t< f ^^"1 iTx 1 1 , • • > Of 'r 1 5 1 I ^^^^t 50r» rria t er i a 1 . • 

b" " • ( 

I'^nr? ♦ ■ 

V'-Tt ct:r L^m tsp'>Mfic iM":e*r>, t^^'f -fts of kevo, and 

iVif^ appj ' f rxate char f*. '>r t>»blc*:., wu mu^^ 
♦ ab 1 e ' ^ acc f » r :^ ^ e 1 y cu 1 ? * he d 1 mt »f r~. i onn i or 

fi'.^ ;ta "♦Uf^ tit - i.' -'trjy io Ui*' ' pt^ ► . f ibf d di'ii<ir»:a* »n 



phtagmpH ^ofi zyt appQjxJL^ 



r — 1 



ERIC 



79 
82- 





rather;: 






than ' 










' ■! ' ■ ■' 1 




1 , , if^n ..1 




. 1 










• 





412 
413 
^414 
415 / 
416. join 



5^417 
fl8 
419 
42S 
421 
422 
42 3 
424 
425: 
426 
>27 
426:. 
42.9 
430 
.431 
435.„ 
.433 
4:^4 
435 
•43^ . 
437 
-4o6^ 
■ 43-'^ 
'.44,i§f - 
441. 
442 
.'443^^ 
444 
"445 
.446 
''447 . 
44$' 

449 
450 
451 
452 
453 

.454 
45 
45!6 ■ 
457- 

"4518 
45(9 
4^>?r 

463.. 



chord 




fin eKample for a 3/8 inch lieyv^y in a 1-1/2 inch shaft 



4D 



4*1-1/2 



at 

write 



arrow 
u.>r i*t<* 

'at 



un 1 1 
term . 

at 

»Air i te 



T^ i o^ttjoiild be th^^ amount we have to add 

l'> ha 1 f the th 1 cjkii^ess o f t Ke "1^'*^*^/ t.q/ get the ' 

FULL DEPTH OP- CUTr , ''J ' ' 1 " ( ^ 

7. , 216; 96, 216;-.]' if>; 1 IS , 1 b ; 1 76 , 5 1 6 ; ^kipf 208, 2 1 6; 2481^,216 

r^'^oa ...... . ' 

■^\f 'et M.ioul ri be oLfr tv>tc^l depth pf cut for; the 
3 ' ke';,^'i.c>y that ,.1.^ u^^ed in thio exarnple*"^ 



.Ml,, 

B<f 1 f me . That meo ^-u r et nent 1^600 1 d be t he " ' amoun t t hat 
u.t'i vMouid s^et on our ma.chine for Lhe FlJbk*.Wep!:h of cut. 

r^l] I 

Cn d Vow forget .the DE!CIi1it-jL point"? That^ Little dot 
jr. ver V' i mp0r t>5 l^t .V 

^r^.sf^ta ■ ■. . ■ ' j\ ■ \ ; 

•f^idd half the- key thick]n^;3:3 .1675" to the j chorda I 
e-iigrrfent ,i'i2 34" that la sV\own above. 1 

p'.:iuw - ' ' ■ ' . , n \ " , ' ■ ; 
1 : \ • - 

UMC<"i the - 1 d-t-pt/h' of cut ha3^ been rnr^de on the 
I.? chine, niu'r.t ir^ke into con^a d-^rat i on exagfclv^ 
licw u»e ar-?^; goinj> t:o me?^^ure theXdepth of the 
ktriyiuay, , • . \ . " _ 

There are two oa^^r^io ways of mea^*wriTTg-/the depth of 
the key" ''^y csnd er^ch method tA> i l%>,^iYe ,ai different' 

, readinp^. If we niea'sure from" the top of tl'ie o£ tVje 
/f.V*a f t t o t e bot t om ^t-' U le .key< ^»av^ * we j a?- fl^ea Aur • i r j^i^ 

/o'H'^A fun ,^ >th ci.rt . ifh'*^^ dopth of the *.keyi«J-.ty pIu^V 

jji^'he \^hor4':*i, e$^.>ii :-Ttt) . ^ . w , " 
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■ ■■ \\ ' ' ^ ■ 

part~l, bloc\^g — v* — — ~ 

. \ \\ space 



U4e .SUB,SUP td'get l x f i 
tint 529 Z6 nice,. 



Goodl^ 



Tht (Otali^g that acjcofn- 
fXirCtti thli txploLnation 
X6 vQ/iy ^pJicAjd ---^ nice 
mkkl ' 




By p I'ac i I hd" di^pfl, h f n i ' ; r om- 1 i: er crossv? j: he t op o f 
klie key way and n.t'=55jur ing to th^;s bo t lorn, are only 
m'^a^suririB the. depfh of t'he keycgay. - " 

If ,we ap*^^--m?Hrt:^.uY\irii: fr<>m \:Y\<?, top of the f-haft to ^he 
bottom of the key ^ gay, i'»Vjaiy«*h'juld be our correct 
measureuf :f »t*' 1 f we are to the FUl i. DbWH rnachirfin^ 
a 1/4" .V-yway on a r'.;.hoft*? 

Th^ formula for chorda 1 3^^geml H-^^p, =.0156" / 



/ 



/ 



475 arrow 



47 enpy 
A77 atl 

478 ^irjite 

479 ' 
480 
'481 
482 



"xtnow 



ric 



^83 

no 

485^ at 
486 ,v^>rite 
487 

43'6 * 

439 " 



5J£f6 unit 

5BT t*:srm 

508 h^lp 

Biif 9 &i 

,510 uir'fite 
511 

51*3- 



, i4j , ,m2& ■ " 

By addin^^ 1/2 th-G width of the key i, i25) to bhe 
cViorcfal 3^.^^Fnent (,0i56) you. get a total '.'i^'pth 
of /:::ut of . 141 " . ^_ ' , . ♦ . • . 



/ 



ite Did you forget Ihe decirft :tl poinr? 





T,> calcu.ate the , TOTf-'iL DEPTH of CUT, Take HRLF the 
tf^e thiekr<es3 of the key and a^zld that anion nt to 
the CHOK'FiL SE01ENT. .>5tf>2=.i25' ,125+ .i3jj56-? 
find the total dept}^ o^f cut, 

■0 * ^ 



oqu8 
t.-b1e 

liy 

T5. you ^calT^Bee^the^'chof^T s'^^gment must be propet-tiy 

1 cu lat ed i n orde?-- to acVi i eve t he jneces :3ar y acci j acy . - 
The ^i.har'E eb6ve has been prepared for you, to elirij:nat< 
having- to ca.)o.<Iate the cViordai ae^ment. 
box / . 

?f}C^4 ■ , . , V . ■ 

The lefi: hand column indicai-,cs ^ he diairiTrter ofHhe' 
viha f t . The ox her c o I u m »Ys i nd^i ca 1 the w i dt h of 
the keyj,*.«ay to be used. " , " > , 

You may refer to tHis cVk-.^ t when needed by pressing 
the -HQ.P- key -on the rissV^t tside of your key^^et* 

squ^ . y 

. table . : " \ . ■ . 

4i59 . *. , ■ ^ ^ 

There are. other ways to rriuasure the depth of our 
keyviay, than using ,a depch mioromete;r and measuring . 

frorfi the shoulder t<> the bottom of the key. ^ : — 

rrtike . , * ' 



Thz 6tudQ,nt YimoA pKaeMaoi 

be fee m;.c££ npi need . 
to tojouiYi thz iowuM on. 
cjoiZauZatQ^ uaamq 44p only 
that fee ta6££ nazd to know ' 
hom^ to u^e ifee KZ6uZt6 oi 
appJtytng tkz ^omixixi. 



CovJtd you put a dAjigmm 
hoAz li not aMi^QAod 
..don/iQ^atly? 



part=l, biock=h 



Dfc-ace 



t?oei6 ifee 6tuda'Vt nzzd to 
pHjOL^tlCQ, fizaduMQ tht 
tab^? 



VoiX coutd put -hzlp tabln- 
tn tho, main UYut at tkU 
po0t^ Tfee/te ^ no need 
yto aHango, youA tu^on, 
homv2Jif ju6t A.mmbz/t:JiJt 
^o^.^'thof niixt tmo,. 



join 
do 

•pause 
write 



unit 
'term 
help 
base 

at, 

tt)r ite 




next 
liirong 

at. 

wt^it/^ 

no - 



smike • , ' " 

box ' • . ■ 1 * 

<at, 240-8>When<5Ver we have, a-nreci^uring device that .will fit 

into the key way, such. a. s an outside m i crorfieter o^^ 

a vernier callipers, we <z^i^ measure from the ; 

bott'cmi of the key way to the bottom of the shaft.! \ 

By '.knowing what this measurement should be, you ■ 

ca^ accurately measure the depth of key way. i 

oqu9a 0 ■ • 4 . 

squ9a • 

table ■ ■■ ■ ' / 



210. ' ■ ■ ■ 

Let 's .a-=v'::.ume that you are to c:ut a key way • 

fer a - 1x4"^ square key m a jI^^^-tT25) shaft, 

'w^f^^at would be the T^^6\r\^ from the bottom 

of the key to the bottom o*f the shaft - if 

i j you 5irT: to the correct * depth? 
» « ' ' y ' 

RrilFJIBER the CMORDRL SFGfiEHT^ ^ 



yo.u noe 



:{d to^use the chorda 1 segment chart 
so by pre^^sing the ^HELP- key... 



sfM \ ke 



4,28^ 
64,2139 

2>rC 



<at,260r,24 

^^ici3 ■ ■ " .' ' ' ' ' 

Real good. By ;5ubtr acting the depth of the key 
^nd "^Vte ohardai ' ;i'.e(^ment .(.0139) from the diameter 
:> I t he 3> la ft "j^ie 6bt a i n h Vie connect ' read i r\% (.986) 
proper depth. * - 

v.^:0= 1 , test ,y ' ' ' 

?S6 ^ • . ' " i ' 

Did you forgefc the dec una] point?- 

You must first calojlaie the TOTRL DEPTH. of cut, 
by taking4"IBLF the key thickness \.2S^-^2^ .125). 
.125 pl.us the chords 1 se^^ment .;5t39 - .1389 or .1; 
Then subtract .13'9 (r/jhole depth) from the shaft 
diameter 1, J 25 1.125'- .139 = ? 



(. 1 25) 



Good ii^z SUB/SUP 



Vw, good to )immi 
6tude,ht6 fjohm -HELP- jU 
aatlvz. 



Vou can aid d -tatowgu 
to put tkt 6me^^te/taMt 
oyi yoa/i ahe,ak ion. dzcAmJi 

QJUlOKJ(>. ' 



— part=l, blocks=i 



space 



559 unit 
560^ term 

561 at. 

562 writ€» 

563 ' 
?d4 " 



fldt .< / ^. " , 

fiat, . . ■ ■ V ■ " 

.finothet^t ype of common key* is tV^e FLAT KEY. This key 
ir> sijTiulv\r to the SQIJFiRI.. KgY but has some different 
'cV-ar act OTIS*: ICS, one is that the key is r'^rctangular ' 
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671 calc 

672 at 

673 xwrit'^* 

674 * 
675 

676 ^ 

677 arrou^ 
670 ansv 

679 at 

680 write 

681 no 

682 write 
663 \ . 
6S4 

685' ■ 

6ee 

6^7 

688 

689 
• '690 

691 

692' 

693 

694 

695 

696 

697 

698 

699 
- r;7.y0 

701 

702- 

703 

704 

705 

706 

707 



unit 

at 

wnite 



do 

pause 

at •; 

write 



do 

pause 
at 

write 



708 unxt 

709 . term 

710 at 

711 . write 
712 

713 arrowi 

714 specs 

715 answer 



. yl00*vl0k5> 1 . ^ . 

1910 

Real .go<>^f. Now that you have accurately ca leu lathed 
the correct depth of cut you .should also calculate 
the corf^ect micrc'ineter reading from the bottom of 
the' key to the bobtom of. the shaft. 
.2410 

v89, .00^20 . V / 

2510 , * 

You ar^e doing great 1 I 

You- can ^et the correct answer by 
the total- depth of cue from the ^shaft diamefcer 
wltich 1 . inch. 

rdl " . . . ' 

rdi . . '.^ ♦ - 

803 • 

We have diso^'ssed two of the most common types of 
keys that, you are apt to '.encbunbiir as a machinist. 
Another common type is the round end key. This 
key is also like a square key,, in that the width 
• and the thickness are the ^same* . 

box 

1408 .o . 

One or both er/ds may be rounded, depending on the 
intended use of the key. The radius on the end of the 
key is equal to one^^Thalf the width of the key. 
fn\, the dimensions are exactly the same as for square 
hey when you are ca i cu 1 at i ng the tota i depth o f . cut . 
7h^- length of the key Bhould be ^qual to a minimury^ 
\:.f 1-1 2 . t.xme-c. the' width of the key. • 

. box . 

2208 ' " ^\ 

Pound end ^eys are used pr i mar ly"* when the mating 
pf!irfs <^»^^ lopated away f^om the end of the shaft. 

rd2 

0808 ■ ' ■ i ' 

To produce a' round end keyway in the middle of a 
sihaft, what type of cutter should.be used?- 

Ill 08 i 

bumpshi ft ,noorder,okexi:ra,okspel h , | 

(end mill, endm 1 11 ) < two 1 1 pped > 



SkouM iinit bdqm 



See SUZ/'SU? (iomznt on 




- part-l, biock'=k -y-r- 

an6W2A judgAjng/ \ space 



716 
717 
718 

'7i9^ 
720' 
.721 



er|c 



hetp 
at 

write 



table 

1308 ' * . 

Exactly! Rn end mill" is the most 4^ppropri ate "tut ter 
to Juse, The size end mill to use must be the. iam<-i 
as the width of the key. Ulhen cutting the keytltay 



we sho.uld allow approxiftiat ly .005'' to 

83 



.006' 



lYtone 




This chapter describes the philosophy and technique of 
in-progress" review, discussing some of the inherent problems of end- 
/ .0t-lesson reviewing, some* past experiments with review techniques, 
:inJ our current review methods. 



85 - 

* 87 : 



13 - THE p^IOLOGY OF REVIEWING 



A lesson review can be one of the most valuable tools an 
author has for writing effective lessons. Experience, however, . 
-has taught us not only the usefulness r^but the precariousness of 
reviewing. Oftfen the value of a review is clouded by factors 
which are interx>70veh with but irrelevant to the review itself. 
Authors, are sometimes prey to pitfajls inherent in the process 
rather than the product. 

In the past, wfe've had scant success at getting authors to- 
actually use reviews to improve their lessons. While authors 
were always perfunctorily complimetitary , they seemed more, in- 
clined to file reviews away, giving little indication of what they 
disagreed with or objected to about reviews. Assuming (per*- 
haps naively) that review quality was not the question, we 
concluded that other more subjective or ambiguous elements were 
involved. The fact that authors- were sp reluctant to talk about 
reviews seetned to support the assumption that feelings rather 
than issues, were at stake. Thus we turned our attention from the 
review itself to review techniques and from the impact re- 
views had on lessons to the effect they had on authors* 

This, then, describees our experiments in exorcising both 
the technical and psych/^logicaT^^eiiTon^ in reviewing,, 
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PSVCfWLOGKAL TPs/\PS 



An author makes a considerable psychological and- emotijxaai^'"^ 
investment in his lesson. Just as artists . sometimes find 
it difficult to separate themselves from their work, authors of ten 
find it difficult to separate themselves fr(»ir. their lessons. To 

criticize the lessor is to criticize the author. Conrsequfently, , • 

/' 

the reviewer Is ofte^i regarded more 

. /.■ ^ ■ ' . • 

■ ■/'■■--.; 
as an adversary t^an an ally, parti- 

cularily when the^r^iewer lacks in- 
structional expeifience. I£ the two 

- 1 ■ \ 

have never met, ajn author *s image of 
a reviewer is frequently fashioned by 
factors that .have less to do with the 
reviewer than with th^ author's own 




Thfif reviewer is often regarded more 
as an adversary than an ally. 



i 

i 

frustrations. 

/ ■ . . . • ^ ■ 

\ Apart from the question of author/reviewer rapport, there 
* ^s the problem. of time. A thorough end-of -lesson review typi- 
cally requires^ number of weeks to prepare. During that time the 
aii\thor has usus^tlv begun work on a new lesson. Since 4nthu- 
^,^iasm for the old lesson is usually replaced by the pi:4oc- 
cupation and momentum of work on the new lesson, revision is rele-^ 
^ gated to a limbo status to be carried out "as soon as tHis new 
lesson is finished.*' As the old lesson gets "colder,** revision, 
seem^ less and less important until it's easiest to regard a 

lesson as "finished" simply because %'4j^lng is complete. 

V V, ; . ■ 

The most detrimental drawback of t»he reyaew process, however , 

*^is the problem of the part versus* the whole* Any task is^ more 
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. burdentioiTte hsindled collectively rather tnan selectively. 

The impact of a review exemplifies this/ pitfall* Singly, 
; each criticism or suggestion the reviei(7er. makest may seem reason- 
able and helpful- to th^^ author. Collectively, however, the sug- 
gested changes appear so massive that/ le'sson revision can seem 
overwhelming. The author is likely /to regard the reviewer 4s* 
overly critical and pimply shelve t^e review entxrely. 

TlCiimCAL TRAPS ^ * . . * 

Two technical problems center around one outstanding 



dilemma — time. At the sites we work with, authors are under con- 
siderable pressure to meet semester deadlines, project deadlines, 
etc. Thus, they sometimes feel an understandable reluctance to 
**waste time*' revising, what is considered to be a completed les-- 
9on. / 

The reviewer might also find himfeelf suggesting majoj: 
ipf'structional strategy changes which essentially amount to a 
^^omplete overhaul. The need for such .^ubstantial revision, 
^/ apart from being time consuming, might ^ave been averted ha^d . . 



the author and reviewer been able to consult in the plan*- 
ning stages, using the review as a proposal rather than a post 



mortem, . * . 

The problem of author/reviewer consultation is also a tech- 
nical stumbling block. A review, almost by definition, is a 
ratherl lengthy soliloquy, a one-sided excursion through the 
lessor/' s strengths and shojrticomings • This monologue quality > 



coupled with the other inhibitive elements of the review process. 



does little to enhance any real exchange of id^s between ^ V 
author and reviewer.. 

Realizing the obstacles, our task bee am4 a simple matter of i 
developing a review technique that removed y^^^ stigma of reviewer \ 
as "critic/* reduced the author *s emotional involvement , capi- 
talized on lesson momentum, minimized both the impact of and 
— ^ ^^^d fo^* massive revision, made" eff Icierit use of author time, 
. and established a free flow of ideas between author and reviewer! 
" Our initial review experiments matinl^ dabbled with vari- 

atiorts on the end -of --less on theme. Though authors were as com\ 
plimentary as ever, there was still liittle evidence that they 
actually use4 the review for lesson revision. A cleaner break 
with our old techniques needed to be made. 

IM-PROGRBSS REVmmiG ' . ^ "^ ^/ ^ " :^ 

' ' ' 

An in-progress review, like an end-of-lesson revie^^is 
set o£ comments used f to improve the instructional effectiveness / 

in-progress reviewing, however, the^/eviewer. 




of a lesson. During 
examines the lesson in bits and pieces at various stages of devel- 
opment rather than as a whole after coding is complete. To main- 
tain continuity and an overall perspective, xn-progress reviewing,- 
rather than being a fragmentary approach, is a cumulative pro- 
cedure in which the reviewer te-examines "old** sections ir» addition 
to successive *'new*' segments. This patchwork process* seems (in 
our experience thus far) to falnimlae a number of previous prob- 
lems. Dealing with the lesson in pieces as it*s being writ, ten: 
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• reduces author def ensiv.eness by examining the lesson* 
^ before the author's subjective Involvement becomes — 
too<»-strong* • " . . - 

m ' capitalizes on lesson momentum by giving the author 
immediate, feedback while the lesson is still* "current 

■ ' ' f ■ ' 

m' enables th^ author to make anynecesss 

"tions or corrections before tliey Isecome habitual. : 



risT 

r 



Though the review process had been radicalj^^tra nsf jOCTgd^ 



we still l)elieved in the iriportanpe^'^ providing the author with 
a^ harldcopy critique of the lesson,. To maintairi the fast "turn 
around, immediate f eedback^poilQy v^e were trying to establish, 

we put generalized comments in a note file on\line» " Then "^^^ 

mailed both additional specific suggestions and an edited print- 
out to the author . ^ The-^only problem was that since the author 
was continually revising' and expanding his lesson, the written 
comments were outdated by the time they reached tiini. 



PRESEA/r 



With the advent of the Varian copier we are now able to 
provide the author with immediate on-line commentary which he 
can copy at his own site* Currently," we put most written reviews 
. in a r */iew lesson in which the authct^ ij^,>also encoutaged to make 
replies or rebuttals, ask question^, etc,- There are, of cc^tdrse, 
pros and cons to be weighed in using such a I'public" tneans. of 
criticising lessons. ^^4ijce authors at some sites have .had little* 

^/ 91 • • • _ * . ■ / . 



CAI experience, this method allows fhem to learn frpm each other's 

successes and "faiiures. " However, depending on tAxe quality of- 

the lesson and the nature of the review, some authors feel jeisti- 

I flable pride over a ''good'* review while others fejel equally justi-- 

fiable embarrassment over a "bad" review. Too often the review^ 

file has (among authors) been a source of competition, rather than 

learning. PLATO Vs monitor mode option now sif fords the reviewer 

and author- tha..!!priA^ feel more comfortable 

with. Usihg this„ technique, the author monitors the reviewer as 

-the two go through the lesson? together frame by frame • Apart 

from "protecting" squeamish authors, monitoring enables the 

reviewer and author to have a free-flowing, spjE^ntaneoiis exchange 

of ideas, viewpoints, etc. * - 

When^a particular change is difficult to explain and the 

. • author needs some clarification^ or when the suggested altera- 

- — ^ion is time consuming (or beyond the author^ s coding ability), y 

the reviewer copieg^jart^or all of the lesson into a workspace. 

He cSn then revise this replica of the; lessotrT^i^hiOLUt tampering 

■ ■ - • , ... * ■ - 

with the original. From the reviewer's standpoint, it 's more • ^ 

■ • • ' ^' 

persuasive to show than to tell about how a. particular phange 

should look and the author, of course^ has the option of copying 
the changes into his lesson. 

FORMAT . ^ " . ' ^ V - ' ' • 

■ ' • ■ \» I ^ 

- - - * • ' "^'^"'^.^ . • . * \ ■ 

In-progtess reviews cover Bfte-s<^me areas and lev'^els as end- 
\ of -lesson reviews. Whereas end-of-lesson reviews are generally 
• more lesson oriented, in-progress reviews are more author oriented . 
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« 1 



The reviewer tries to be sensitive to the effect his retjlarks • 
will have on the author -and to temper the tone of the teview 
accordingly. Since our working situation is such that we deal 
repeatedly with the some authors, our reviews are o£tet» directed . 
more at long-iange philosophical changes than at s)f)ecif:ic lesson 
changes. • . 

• A written in-progress review usually opens with general 
comments, about the re^j^iewer's. overall impressioni. The. reviewer 
tries to point out those parts he^ parti<:ularly likes, any tech- 
niques that se^m^specially eff^tive, etc. 

- The"^'Bdc[y~t>l-r^e_^e\^ is a unit--specif ic assessment on a 
number of different.^.ieve% (see section. Sj^ome of which are: 

A. Details * 

\ 1* spelling \ ' 

2. punctuation \ 
.3. grammatical err(^rs 

B. Display techniques ; 

1 . highlighting : 

a* underlining ] " 

b. sizing 

c. * pausing 

2 . characters 

3. positioning 

4. amount of text. 

C. Content 

\ 1. lesson flow * 
2* br-anching 
3* contradi^ctions 

4. conformity between Content and objectives- / 

5 , style > ^ ' ^ 

, ' D. Teaching stratej^y 

1. interaction r ; ^ . . ' 

2. graphics / 

• . 3. simulations 

4* questioning Jfechniques 



■m-PRQGRES^ VtRSLL^ EW-Of-LESSOH REmmm 



i^n summarising which type of review ^ould be used at what 
times, a\ number of points should be reiterated. 

. ■ . . \ ■ • ■ ; f ^ 

in-pr6^ress reviewing is art excellent technique for giving ^ 
immediate .fee^ack. Thanks to the system "talk*' option and 
moi^itor mode, thexrevlewer can fit the review to individual 
author needs, and reviewer arid author are able to work in close 
jrapport* In alddiLion, the reviewer figures more prominently 

*•,"*' . ..' • , . 5[ ' . 

in lesson development and his, role is broaidened to include re- 

views of education^al objectives, 
criterion test-Sj^-esson design, etc. 

There are' two main dravrbacks to 
in*T[)r ogress reviews, however. First, 
though reviewing lessons in sectiojis 
reduces iauthor def ensiveness arid 
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The reviewer often reocts os sgbjectiyely maintains lesSOn "momentum,'^ it *S 
as the^authoi*. 7. 

djlfficult for the reviewer to assess 
the flow of the lesson or establish much^ of , a total .view* . Sqcondy 
: after r^eviewing the same lesson over U^^eir^io^ ^f weeks or moriths, 
the reviewer can tend to lose^ftis^obj^^f^l^^ and independence, 
often reacting as subjectively toward th^; lesson as the author. 
For these reasons, ^fter'a ^viewer and adtl^pr -have worked together 
on a lesson and each considei^ it /'finished,** we have a second re- 
viewer do a final, objective end-of--l^ssson review ^ • 



SAMPLE RfeVI^ViS , ' - ' \ - 

The fallowing siectipn contains review samples and ^'faefore^j 
arid after** examples of some sorts of changes one author made in 
his lesson after a review. I * 

Sample III is a review of two sets of Wesson "objectives* 
The objectives to be reviewed (with openings ^uthor comments) 



are followed by the- reviewer -a stig^estians*. „ . 

Sample IV is a "follow-up" review of a lesson which had^ 
been .reviewed numerous^ times already at various stages of devel- 
ofKnent. The numbered comments all liefer ence spetci^fic, units* 

Sample V includes a "first" review of ^ lesson ^Ith examples 
of some of the changes the author made as a rfesult of- the review. 




"Here; are the lesson objectives 
and ^11 comntents are welcome. But,N,I 



fax. t;he topics listed. Any 
djon't have a flak jacket on.** 



RESP MySCLE PARALYSIS 

Iv Student must identify two types of neuromuscular l^locking agents and| 
^ must define their mechanism of ac/tion, 

2. M^Then giveti the name of a neuroitiu^cular/b locking agent, student must 
determine which type of blocking the drug represents. 

Student mQst define structure and fxinctidn of a neuromuscular 
Junction, j „. ^ . 

4.x Student must listi three measures available to prevent botulism poi- 
soning. ' ' " J . 



ObjaztAva ^1 ^ voky good. Vo^ ikz vjond idtwtiiij mojun that j 
tht 6tjuidmt ioM ^ttat^ on. pick oat tkn tvoo typo^? In otkoA woKd&, o/te; 
you (UfUng tha 6tudznt to mpply on mcognlzz in^o^matijon? 

Z. Ltst and dz^im cuit zxcoIZznt bzhavionj^ havz ^pzcl^A^zd in 
quQ^txdn^ 3 and 4...kotv, 6p^(Ufically, a^e you going to ttu^t thi^i^ 
^onX 0^ bzkavioK? \n.z you planning to cc6e qut6tijon {)om^ otko/i tlmn, 
muUxpl^-ckoicz? 



NARCOTICS 



1. Student must list three characteristics that Indicate narcotic 
analgesic intoxication. 

2. Given the definition inflammation of a vein, the student must be 
. able to determine the condition described. . 

3. '^Student must list trade name and generic name for one oOt of three 

narcotic antagonists. > 

96 . ^ ^ 



Sample 
Reyiew III 



bQction, too. ' 

2. Objzctivz !3 lb vdiy good, v2Ay p^eci>4e,. ' ' 

4, Student must pick . narcotic antagonist of choice from a list of 
four drugs. ; 

5. Given^our statements about narcon, studfent must pick, incorrect 
statement. 

3. M(/ lack o^ 'cont^nt knoiA)lzdgz aj> 6homng agcUn, but ^4 4eem4 
to 6e m^i^^Vig ^oimthlng . . . do you mzan ^omeMUng LLkz, "G^ven a numbeA 

naAcotic6, tko, 6tude,^it mu^t choose: tkz ayitagonyUt o^ cholaz ion. 
cack ^Mom a LUt o^ iou/i d/iug6?" 

4. Objzdtivz 5 x4 ph.zcJjizJiy stated, but donvayi .\J2Ay littZz about 
LiiiiX^t jzxac^ij you mnt tkz ^tadzwt to knoi^ about noAcionV Von, l^htancz, 
ojiz you talking about akmlaal aompo^Mlon, zU^ct6 pKoduczd, antagoyiUt, 
y^Xc. ? , • 

Slnad ^o.many objZcXlvQJ> Kzquviz tkz ^tudzvit to Supply In^onma- 
tion {dzilnz, IdzYitliy] ^natheA tlian dimply Kzcognlzz ^inionmatlon, 1*11 
bz \:fzn.y IntoAe^tzd In ^zzlng tkz cAUX^on tz6t to 4ee hovo you tz6t thu 
\^bont oi bzhavloH,, • * 

That* 6 alZ ion. now, . 
/ Sa^cha - 



97 

98 



Review^tV 



"Uliat more can we sav? This series of l^.^sons has matured into 
a truly impressive example of how to be both interesting (and often enter- 
taining) and informative at the same time. Though it takes quite ayhile 
CO work through the entire group of lessons, I don't think you'll hav^. 
any trouble holding the student's interest. ^ 

A number of things stand out in pur minds, and deserve recog- 
nition... the most striking aspect of the series is that you use a 
number of different strategies, questioning techniques, display tech- 
niques, etc; You also have the student interacting in a number of dif- - 
ferent roles. (sometimes as a student, sometimes as aP,A,, etc,) — 
these are two factors which keep the lesson -from becoming tedious or stale. 
Another impressive technique is the fine use of many types of feedback... 
the hints .you give are instructive and provocative really excellent". 

Reading over this, it sounds like an awfully lot of backslap- 
ping, but you really have reason to feel proud of your work on these 
lessons. The following comments are mainly about those insidious de- 
tails, . .don' t be alarmed by the number of them., .remember they cover a 
number (3?) of lessons rather than just one/'' 

1. ''object2" Using different pharactera to highlight impor-- 
tant words or phrases (in other displays as well) is very 
effective. , 

2. *intro2a" This is an old comment, but one I still feel is 
important! The ''Help'* could go to the fiche of the nasal 
structures — maybe with letters/numbers indicating each im- 
portant structure and it's position. There's a fair chance 

/ that seeing the structure would be enough for tfte student to 
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\ 



be/able tg recall its^hame. If he is stjJJr--^tubk^, you*' 
^can then\|ive 1:he accual answer (s), but I think it'^xini- 
\portant t6 rjiinfbyte the. visual associatiion in the student's 
Vind , 




mose" Put/t\iffig the med , hist, ^pn anpther page ai^d moving ^ 
tjhe ■ introduction ,doTO would higtublght the inforn4tion iri 
bjoth s;eclfifons^^ — the med, hist, in\part icular , Since an up- 
ciDming ^^^estio^i deals with whether oV not the student has 

! ■ \ \ * \ t ° 

njDticed the temp, on-^he%iist, chart, >and since a P,A, nor- 
m^lly sees vjthe chart s.epar^|tely , this s)^ems more fair to the 
student, N I \ 

"[choice** Why Isl 'Wke a diagnosis*' the fi^rst choice? • A 
cjouple of other |:hoices seem mi'^ed up — aiW particular 



Reason? 



\ 



VhistryS/* One way 'to eliminate the;, problem of the student 



havipg forgotten 

N i 

genejral , introduc 
comnlents this 
hisl|, part, 

stnl" The dis 
you g^t ther 



how 



some of the itif orm^Jtion is no put all the 

■ \ ■ \ ■ 
tory-type sections before the specific case 



would \mean histryl-2 l^ould go before the 



play acts differently afpend|.ng on when and 
, , ,the fitst time thru, eyeryt|hing at the 



bottom of the page was squeezed together an^ I Impede a ndte 
that the discussion should be moved up and. tl^e jquestion (about 
watery discharge) moved down, ^ then when I cam^ back and went 
thru by condensing the lesson, the display was\exactly as I 
was going to suggest it should be. Some restatj^s must be out 

r ' \ 

of whack,, but tjhe way it looks via the lesson ls^,the best, 

\*questn2" Would the word **normal'^ be an accepjtab^e response? 

•'rp9al** "rplOai" Both talk about negative inf jarmaVion , but 
don't say speci^fically whether it's contributqlry oi^ not. 



Sample 
Review IV 



/'nsalexam" Move over a couple of spaces,,, the first line 
spills over into the second line, I likfe the .help very 
much — much better than when you simply gave the coirreQ,t 
answer. ^ ' 

"nasal2'* It would be nice if you'd keep the answers the stu 
dent has already given when returning from the help slijl^. 

^*nasai3a" Would be nice to advertise some help rather tiiap 
risk having the student sit at the arrow not knowing an 
answer, not being able to get help, and not typing in even 
a wrong answer (and getting help that way) . 

'^nasal?"" Will th^ text ov^erwrite the display? Seems 
to indicate that ij: will.,. good idea. 

"ear2" Maybe you could, state (in parentheses) that you 
want a direction,' even before the student responds. bit 
confusing. 

"ear8"- Needs some more helpful ^f^edB^ck, Says to use terms 
like anterior, posterior^^^c. You often find yourself 
cycling thru the^pegsiKilitdes and getting the same (not . 
too helpful) feedback. . .might be nice to'Wve correct answer 
on ntries. \ / 
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Sample 
Review V 



I know I've said this before, biit.this is a very clever lesson. 
, . ■ \ ■ \ 

You really have a good grasp of programming as well as writing lessons. 

A lot of bright ideas made this lesson interesting and interactive. To i 

be more specific — good use of animation and underlining, spelling 

judging and correction of misspelling, I'll try to be helpful to make 

this lesson even better; I have been pretty critical, Iti general, I 

feel the questioniiig technique can be improved — to ask questions at the 

right difficulty level and measure the true understanding of knowledge ~ 

teaching sheer rote learning is too expensive a price to afford. I ^ 

have made specific comments on each unit but tried to avoid meaningless 

repetitions. You may find some of them rather critical. Since ypu 

have familiarity with the subject matter I also need your comments on . 

the following review, I hope they are helpful and useful. 

Susy 

"intro" Artistic work. You really have a flair for creative ani- 
mation. 

2. '""stuname" Allowing student to choose a name really aids the person- 
ality of this lesson. 

3. "comment 1" It is beneficial to have individual units comi^ented by 

other co-authors or reviewers, I think it provides a good channel — ~ 
for improving the lesson quality. However, students may find it 
more handy to make overall comments at the end of the lesson when It 
is presented to them as a f inished , pr oduc t , ' , 



4. "alobj" A clearly stated index page, 

5» "alobjl" Wondering if one objective is adequate for mastering materials 

J 02 



/ 
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Review V 



In this section. I would suggest leaving out the headline "of 
ALLERGY to avoid redundancy (it is the headline of the previous 
page); instead, give the section name A. VOCABULARY a more promi- 
nent role to attract attention to it.* 

6. *'tcalvoc" Underlinings are very effective. 

7. *'tcalvocl" Unless your idea is to have students memorize the terms 
and their spellings, using the cf ill-in-the-blank type of questions, 
especially when supplied with the right number of letter space, may 
cause a rote learning which does not necessarily require true under- 
standing of the given material. Hint ( anti) given after the first 
arrow also shares the same characteristic. Like the circled "but'' 
very much.*^ - . 

8. "indtv*! The explanation on individualized effect of allergy is 
very clear. Would you think to accompany it with some real cases 
could be more illustrative?*** 

9. '*indivl" The use of arrows *and boxes is very effective. Somehow I 
was a little confused 'when I first looked at the animated message. 
Some thoughts about it...(l) ''indiv2'', *'ralvocl2-' and this unit are 
plosely related, but the way of presenting this basic fact (i.e,, 
atopic people are sensitive to allergen) makes the acquisition of 
knowledge rather too difficult. I would think rewording the text 
in these three units might clarify the relations among antibodies, 
antigens, and allergens; (2) do you feel that a particular class of 
antigen/allergen should be stressed somewhat to m^ke the transition 
of pages more smoothly; (3) wondering whether too much emphasis has 
been put on the last sentence (size 4 writing) compiared to the rest 



_ *See Figures la and lb. 
**See Figures 2a, 2b, and 2c. 
***See Figures 3a and 3b. 
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a. Vocabulary and Mechanism 


1 ' 
\ 




When you complete this section 


\ , • ■ 




1 ou wxxx De aoxe .to use prcL-ist; 


1 




terminology to describe the 


i 

i 

i 




mechanics of allergyJ 


' i ^ 





Figure la - Before 



\ 



Pigure lb - After 



. V0CABULAR Y: AND MECHANISM \ 

~y ' ' '. 

When you comp/ete this section you will be able t0 - 



1, Differentiate between terms used to 
des/ribe the allergy mechanism. 

Differentiate betwe^en stages 
oy this mechanism. 

ame the tissues affected in allergy 
knd their physiological response. 
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Antigens stimulate the body to produce antih/odies. 

> -/ ^ - , —/ — 

Antigens are recognized by the body as / sub- 

i stances / 



Foreign ok - , 

You got it! , 



Note: When the correct response is given the word ^appears in the 
blank space . ' 



Figure 2a Before 



A Child is protected from many diseases by being 
immunized at an early age. The DPT injection 
contains killed bac ter ia whic h stimulate the 
body >tp build up its"^ immune defenses against the 
organisms which cause Diphtheria, Pertussis, and 
Tetanus. 

In this application of the immune system, name 

a specific antibody formed. 
. DPT antibodies 

.This is how the body is stimulated to bu)ild dafenses. 



Note: When the student presses -NEXT- he §6es Figure 2c. 

Figure 2b - After 
104 , 

105 ' 



/ 

/ 



A ahild is protected from many dis^'^ses. by being 
immunized at an early age. The^^T injection 
contains killed bac terTa^TITich'"'"^^^ the 
body to build up its immune defenses againrst the 
/ organisms which/ cause Diphtheria ,1 Pe^rtussis , a-nd 
Tetanus. 




All a,ntig^ns do not affect all people, in the same way. 




.1 



FOR ExAmPLEI 



A group^of people exposed to the same antigen will show a 
variety of Individualized responses. 



1 
2 



Some, people wll/ produce no antibodies, 

/ ' = 

Others will produce antibodies 

but show no. symptoms 

Some people will produce antlbotile'5 

and show symptoms. 



Figure 3a - Before 



FOR EXAMPLE] 



Pictfure this party. I hope you*re having^ good time. 
One of the guests Is having trouble with his allergic 
asthma and finding it hard to breathe. You'll believe^ me 
if I tell yoii hels being poisoned by the room air. 



Mr ^xAsthma I is - the only one to respond to this poison 
which is rdaJLly microscopic pieces of cat fur from the 
feline undeW his chair 



Name the an 
produce ant 



igen which Is stimulating Mr. A*s body tc 
bodies. 

^cat fur 
Splendid dear Watson . 



ERIC 



Figure 3b, - After 
106 
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Sample 
Review V 



of displayed ma erial. The percentage of atopic population might 
deserve less significance (please forgive me if I am wron>) . 

10. "indiv2" Make the flower (symbol of allergen) more activ.a. Per- 
haps it will impress the people better/ That is to flash or place 

^ the flower right after the word of allergen (in the first line) in 
writing instead of the present form. Another thought — ; since this 
symbol stands for allergen and antigen at different times did you 
find any possible confusion or ambiguity that might arise regarding 
the u%e of this symbol describing .allergen/antigen business in 
later units? 

11. *'ralvocl2*' Have some si/|gest ions about the questioning technique. , . 
the f ill-- in^-t he-blank type items may bply measure the acquisition of 
material at recall level, meanwhile, the number of letter spacer 
provided for answers is indeed a hd^nt too obvious to miss^ BUT, tbe 

-answer space for the first arrow may cause the hesitatircwai of filling 
atopic in while the number of letters indicate that it might^tiot 
l?e acceptab^e^ If the spelling of terms can be tested elsewhere, this 
lipit is mo^e appropriate forNy^^ testing the true UNDERSTANDING of^ t;hose 
t^rms. mV suggestions are: )tmve some real examples about differ- 
ent sorts of allergic results and ask the students to identify the 
X, Y, and Z in this mechanism (X, Y, and Z stand for antigen, allergen 
and antabodies or the like).* 

12. '*tcalyoc3" Introducing atopy right after it was mentioned (in 
unit /*indivl'*) may be better . j 

13. **ral.voc3" ^ It is a copy^frame in the sense that the word "inhetitec^*** 
is/a clue and an answer as well. Change, the test item or the form/ 
Q/r it seems necessary. The second item poses another question — - / 
whether or not **exactly** is equivalent to 1002 WRONG — ♦ if not so^ 
leaving out the ok/bo judging and having feedback part reminding 



*See Figures 4a at^d 4b. 
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Type in your answei" f or the space mafked by > • 

Ten percent of the general population are atopic. 

These individuals produce antibodies when challenged by 

4-'^ . . Most people produce antibodies only if they 



are exposed to other 



Figure Aa - Before 



LLUSTRATIONS 



a/ "The party with Mr. Asthmai 

*B. rnf'ant immun iz t ion against^ DPT. 



Type a or b or ajb for tlie il lus t r a t ions $hown above 



\ 



which show a t o-py 



ab 

No - Atopy is not involved 
In immun iza t ion aga ins t DPT 



■ ' / 



Figure 4b - After 
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Sample 
Review V 



students that ATOPY is not exactly a disease, it is an^ i^nherited 
characteristic. This may make it more desirable. . * 

14. "tcalvocA'' Fine work on animation. Good idea of capitalizing and 
underlining SPECIFICITY. 

15. "tcalvoc5" Do you want the ^'student to remember the term Immuno- 
globulin by heart or is it a satisfactory answer? 

.16. "ralvoc4!5'* The use of extreme term (only) in T/F item should be 

avoided or the answer would be too obvious. As to the second question 
instead of usijig the blank. I would suggest . th^ short reply type ques- 
tion with a question mark. Is it also^ considered as a legal answer 
for the third arrow? Again, do you want students to retype or 
simply copy down the long word (Immunoglobulin) to enhance their 
memory of this word? 

17.^* *-tcalvoc". I. feel the changes have been made about the process of 
sensitization are very helpful*: The rephrasing of ^^ast Cell and 
Basophils-^ is good, too. If antibody circulation can be animated, it 
will be more interesting. ' c . 
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Chapter yi - A Practice. Review 

(DeveToifted by Merl Goldstein) 

■ ' ■ ■'. ' ' 

" , I v.. '-^ • . 

Okay, reader, you have a decision to make. This 13 a practice 
exercise for reviewing; it has an "answer Tcey^~-^t the end. v Ydti can 
(1) decide reviewing is"'not interesting to you and leaf perfunctorily 
through the practice review or (2) deofide to take the practice review 
seriously to see how well'you do. If you choose the latter, yoa'lhay 
want to reread chapter II before beginning to review. Oii^-tfie next 
page you will also find ^ short summary of chapter' 1 1./ If you choos,(^^ 
(1) that's all right, too, but you're only fooling yourself if you think 
that you can look over the answer key and later do the practice review ^ 
with valid results. * 

The list of potential problems (answer key) is not a review. i 

It does not point out the things done well nor. does it offer solutions. 

' - . ■ ■ - ■ * 
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Lesson Review Categories ; - 

Planning Stages of a Lesson 

A. The purpose of the lesson; 

B. What con tj^nt should be selected 

C. Assumptions about the student's general ability and specific 



background 




D. The relationship between a lesson and the course of which it 
is a part 

The Design of the Lesson - , - 

A. Choice of J:he particular teaching strategies 

B. Selection of media . 

C. the extent of individualization and differential routing 

D. Organization of the content :» - 
Implementation/Development of the Decisions Previously Made 

A. Employing the chosen teaching strategies "^ixd me^lia effectively 

^E^^-^-^iBG-^ w handling -of— 4:nd-iv^id^t~4^-#er ences-rt^ 
feedback and remediation ' . I - 

C. Appropriateness of the lesson's tone and style 

D. Lesson test, if any . >^ 

E. Lesson flow ^ 

F. ' Quality and qua^itity of stijdent 'interaction 

G. Appropriateness and quality of the questions 

M. The clarity of the explanations, the appropriateness of the 
reading level, the illustrations, and the example^ 



TV. Polishing/Finishing of the Lesson 

■A. Language . 

B. Consistency in the use of terihs, instructions, keys, etc, 

C. Technical quality of the lesson as a graphic production 

D. Aesthetic appesil of the lesson 
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Binary Mumbers 
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\ INTRODUCTION \ 

^ ' ''X ■ ^- ■ \ ■ . ■ 

InformatiorT inside the computer is represented in a kind of symbolic 
code. This codeXcan be considered, in' a sense, the ^'language** of the 
computer. The ^'alphabet'' of the computer system really has only two 
"letters"; these can be represented in many ways, but are usually con- 
sidered to consist o£ one letter used to show that something is ON and a 
second to show that something is OFF. The ON/OFF distinction is very . 
often represented by tl^e two digits I and 0, 

recorder to become fluent in the language of computers, it is important 
to'^l^ow something about the binary number system and the octal number 
systetB. And in order to dp that, it is necessary to have ^ome under- 
standing of what a number System really is. 

The purpos'fe^ of this booklet, then, is: 

1. To describe what a number^^^j/;stj^iiL-i^--4R---te^ decimal 
™_ . — mtmber-^ystiSTfTTilT^we^re all familiar with* 

/ • - . 

2. To explain the binary" and octal number systems. 

,\ ■ ' ■■ ' ■' 

3. ' To show l^ow these number systems can be used td represent 

information within a computer system. 



/ 
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Three 



INSTRUCTIONS FOR WORKING THROUGH THIS BOOKLET 

The instructional method used in this booklet is called programmed 
instruction. In order to learn as efficiently as possible, work through 
the material in the following way: 

1.. Read the text on the page you are looking at. 

2. Respond in the way called for. 

3. Turn to the page indicated and proceed according to the i 
instructions pr^esented. s 

^Mhen- ytJTTTiave finished the instructional sequence, work through the 
self test provided. In this way, you cah determine how well you have 
masterc! the contents of this lesson. Review any areas that you find 
troublesome. 

TO BEGIN THE i^ESSON, TURN TO PAGE FOUR. GOOD LUCK! 
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Four 



As we merftioned before, the equivalent of "vrriting" in a ^computer system 
is the iLse of two ^'letters," different permutations of which can be used 
as cdd^ that are capable of representing, all information processed 
thp^tjgh the system. 

The' information thus represented can, of course, be either quantitative 
or symbolic. This means that the representational systjem must be suffi- 
ciently flexible to accommodate numerical knowledge and ^natural language, 

• . . ' ^ ' f^'' ' 

The coding system used in most digital computers, as "dppoSed to analog 
computers, is based on the binary and octal number systems. 

These two number systems are unicplely qualified to serve as vehicles 
for this task. We will discuss^izhe reasons for this later on in the 
lesson. J- 



GO ON TO PAGE FIVE, 



\ 
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Meiore anything else, let ' s examine our own familiar number system in ^ 
order to see how it works. : 

• , ■ ' • ' *? 

The first thing to notice is that our decimal number system consists of 
ten distinct digits: 



1 -2 3 4 ' 5 
•6 7 8 9 0* 

These ten digits can be used to symbolize just aliout any quantity .we 
refer to by a very ingenius method: depending upon where a digit:'*'sits" 
in relation to other digits, it can take on different values! 

To begin with^ , numbers are considered to fall into different columns — 
and each column has a special meaning. For example, look at the illus- 
tration below: . 



Notice that a digit takes on different meaning^ depending on which column 
it bc^longs to. And all colu\nns to the right of the One on the extreme 
left must have a digit filled in even if it is only a zero to ''hold 
the place. • 

Thei meaning of each* column can be' described * as follows. , 



GO O:^ TO PAGE SIX. 
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Ordinarily, we would read the nijffl6er 5505 as five thousand five hundred 
and five. Now let's examiji^that phrase' more closely to see \what it 
really means. ^-^""^ 









\ 5 








0 


5 


/ 




■J 
5 

/ 


0 


5 


^ 'A 


5 


5 


0 


5 



The digit five in this cpluii^n is 
really a shorthand way of saying 
that we are dealing with fivd groups 
of things, each with a single item 
in it. 



This is a shorthand way of talking 
about: 

5 grou-^s of a single item each 

plus ^ 
0/groups of ten items each 



In the same way, this group of num- 

.bers means: 

5 groups of a single item each 
+ 0 groups" of ten items each" 
+ 5 groups of a hundred items each 



And this group ofc, numbe.. means: ^ 
5 groups of a f li^I^ item each 

^ 4- 0 groups of ten t^^-ns ^arh 

+ 5 groups of a hundred items each 
+ 5 groups of a thousand items each 



In other 'words, the column in which a digit sits determines its value 
almost as much as the digit itself does. How does this work? That*s 
what we will explore next. ' 



.GO TO PAGE SEVEN, 
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Seven 



How are values assigned to each column? Well, the answer is related to 
the reason tlfat elementary school chiltlren are .of ten taught to talk aUout" 
the hMiidreds, tens, and units . columns . ^ . 

Each of these columns'itakes its name from a value of the number 10, raised 
to some power. And the value of the columns increases as we go' f rem right 
to left because Che power to which we are raising the number 10 gets - 
larger as we go from right to left. - 

THOUSANDS , * HUNDREDS TENS UNITS ^(column) 

. ' 10^ . 10^ / 10° ^(takes its 



The thousands colnrau takes its name from the value of lO'^, which eauals 
1,0^)0. ^ 



name from)' 

3 

2 



The hundreds , column take;: it's name from the value of 10 , which eauals 
100. — 

The tens column takes its name from the value of lo\ which equals 10. 

And 'the units , or ones, column takes its name from the value of 10° 
which equals l^. . — 

(^^E: any number raised to the 0 power is equal to. 1) . 
GO ,0N TO PAGE EIGHT. ' 
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Eight 



As you have ^een, the first column on the right, the ones column, 
takes its value from the /umber 10 raised to the 0 power. The next col- 
umn that follows takes its value from the number 10 raised to an expo- 
nent one number higher than the-^xponent in the column that came before. 

OK, now we have a bunch of columns that have specific value^r" But we 
still have to isee how these values relate to the digits mentioned 
earlier. ' 

^ ^/ 
One way to desctibe what actually happens is to say that the digit that 
sits in a given column is multiplied by the nominal value of that column. 
A number, then, is in a sense, the sum of the products of each of^hese 
multiplications.^ , ' 

For example, the number 5505 can be thought of as meaning the following: 



10^' 


10^ 




10° 


1 

5 


5 


0 


5 



5000 +500+00+5 . = 5505 



The steps in the process may be described as follows; 

a\ The first digit.^on ,the right- is a 5. q 
The value of the column it is in is 10 . 

. 5 X 10° = 5 



b. 



c. 



The second digit is a 0. , 
The column it is in is 10 . 

0 X 10 -^ = 0 

The third digit is a 5. ^ 
The column it is in is 10 . 



5 X lO"" 500 



d. The fourth -digit is a 5. 
-The column it is in is 10 . 

5 X 10^ 5000 



e. 5000 + 5000 +0+5 =-5505 



GO ON TO PAGE NINE. 
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In the same'vay, we can think of the number 396 in this way: 



10^ 10^ 10° 



300. + 90 + 6 ^= 396 



The important thiing to remember, now^ is that no matter what number syjs.-^ 
tern we are dealing with,\ the column^are always set up the ;sam« way. 

The base of the number system in question is always raised to a differ- 
ent power in the different columns. And the exponent used in the column 
to the right, is always.^0. Each column that follows Raises the base of 
that number system to a pou^rl th^t is one higher tl^ah;that of the column 
that came before. For example, if^ we were interested in- a' number system 
based'on 4, v;e might say "that the first few columns ar6 4^ = 1, 4^ = 4, 
42 = 16, etc. Notice, that in each case, we have translated the^Value of 
the column into a decimal number. Then we multiply the value of that 
column by the* digit sitting in it and add these to get the decimal 
equivalent bf the number. , V « - 



GO ON ''to page ten. 



Notice that the^ decimal number s^tem needs only 9 digits plus the 
zero because the number value that follows nine can^be represented by 
putting , the digit in the second column from the right, which has 
the value 10V= temf^^'^=r^. 

Therefore, in a' number system based on twelve, instead of ten, we would 
require eleven digits in addition to zero. Since we have only nine 
digits in addition to zero, it would be necessary to invent two more. 
For example, we might arbitrarily say the symboi follows 9 and the 
symbol '7" follows // ' ' v - , . . 

The base twelve, th^n would be said to use these digits: 

1 2^3.4 5 6 
^' V ' . / ■ - 7 8 9 it / 0- ' . 

On the other hand, in a nmnber system based- on eight, we really need only 
seven digits in addition to zero. That means we- Can do without two of th 
digits used in the decimal number system. 

Thus, "the only digits necessary in a baise eight system: 

' ^ • . " 1 2 3 4^ 

. ■ . ^ - ^ ^^ ^ ^ / ^ ' ' 

Now you try, ' How many digits do you think would be necessary in a num- 
ber system based oA FOUR? ' ^ • , 

WRITE YOUR ANSWER DOWN. ' THEN GO Off TO PAGE ELEVEN TO COMPARE YOUR 
ANSWER TO OURS. ^ , . - , 



Eleven 



In a number system based on FOUR, we really need only 3 digits in.. 
addition to zero.. 

Therefore, the digits- in a number system barsed on four would proceed as 
follows ' ■ ' ^ ' - 

• . i' 2 ■ . » . " 

GO ON TO PAGE TWELVE.. . . . 



126 



vERlC 



124 



Twelve 



Now that .have talked a bit about how a number system works, we 
are ready'^lco turn pur attentipn tio the BINARY number system. 

- -/^ ■ ■ ■• 

The binary nunfber system is based on the number TWO. Since that is the 

case, how many digits do you think are needed in the binary number sys- 
tem? . , • 



miTE YOUR ANSWER DO\m. 

Answer to ours. 



THEN GO ON TO PAGE THIRTEEN TO COI^ARE YOUR. 



V 



12/ 



125 



Thirteen 



* The binary number system ne.els only one digit in addition to zero — 
Sor a total of two digits. - 



These two digits are: 1 0 



GO ON TO PAGE FOURTEEN.' 



/- 



ERIC 



126 

128 



Four teen 



We have said that the binary number system requires only two digits, 1 
and 0. At the sam^e time, values are represented in the decimal qumber 
system — that is, by looking at the relationships between the digits and 
the columns they sit in. - 

And^ since we are talking about columns again,' this might be a good time 
to determine the values of the different columns in the binary number 
system. 

To do that, we have set up six columns in the space below. Your job 
is to add the exponent to each column in order to show its value. 

Notice that the first one has been done for you. 



IVRITE YOUR ANSWERS DOWN. 
ANSI^RS TO OURS . 



THEN GO ON TO PAGE FIFTEEN TO COMPARE YOUR 



127 



129 



Fifteen 



The columns should be headed as follows: 



2- 2- 2- 2- -^ 2- 2- 



GO ON TO PAGE SIXTEEN. 



130 

. 128 



Sixteen 



Now that we have labeled the first few columns in the binary number 
system^ we are ready 'to look at their value as translated into the deci- 
mal number system that we are most used to. 

To determine the respective values of each column, look at the material 
below. \ 



2^ 2' 2° 
( ) ( _) (_) 



Decimal values - 

As you can see, the decimal values for the >firSt two columns have beeti 
filled in for you, ; 

The decimal values were computed as follows: 

2^ = 2 

Now it's your turn. What would the value of the third column be? 

a, 4 o ' ^ 

b, 0 

c, 400 



CHECK THE ANSWER YOU THINK IS CORRECT. THEN TURN TO PAGE SEVENTEEN. 



131 



Seventeen 



The correct answer is a. The. value of this third' column is 4. 



GO ON TO PAGE EIGHTEEN. 



130 

132 



ERIC 



Eighteen 



Here is the complete set of values, 



2 2 2 
(32) (16) (8) 



2 



2 
(4) 



2 

(2) 



2 

(1) 



Just as irtr^te— dec imal: numl) e r sy s t enTpvygncra n r ep r esent ^dlf f e r e n t quan- 
titles in the binary number system by multiplying the value of that 
column by the digit that "sits"^ia that column^ The resulting quantity 
is expressed as a decimal figure. 

Since the binary number system uses only two different digits — 0 and 
1 — we can think of these digits as an Indication of whether or not 
we should "count" the columtr in question. 

If the column is to be counted, we use the di^t-.^^-----^^ 

If the column is not to be counted, we use the digit 0^. 

TURN TO PAGE NINETEEN. 



0 



131 

^ . 133 



Nineteen 



Now see if you can represent the quantity 2 binary. Fill in each box 
with either a 1 or a 0, 



2' 2' 2^ 2' 2° 



= 7 Decimal 
Value 



WRITE YOUR ANSWER DOWN, THEN GO ON TO PAGE TWENTY AND COMPARE YOUR 
ANSWER TO OURS, • . 



f 



ERIC 



134 



1 



o u 



Twenty ; 



Look at the alternatives below, Follow the directions "f oar the one 
that best kgrees with your answer on page nineteen, 

- ' . . ', .\' * 

1. Your answer contained one or more O's in the first three 
'c. columns on the, right, no matter what was on the l^^ft, 

• . . - ■ ". X- 

">GQ^ TO PAGE TWENTY-ONE . : \ 

2. Your answer Xbntained one or more I's in the three columns 
on the left^^i na matter what- was on the right. ^ 

GO TO PAGE TWENTY-THREE, 

3^ Your answer contained all I's in the three columns to tlie 
right and :iil O's in the three columns on the left, - 

GO 1^ PAGE TWENTY-TWO, " , , 

4. Your answer contained all Qv's on the right and all I's in 
the three columns to the left, ^ 

GO TO PAGE. TWENTY-FOUR, 

5. None of the above, . 
GO TO PAGE TWENTY-FIVE. 



FOLLOW THE APPROPRIATE DIRECTIONS ABOVE. 



135 



133 



, , Twenty-one 

If your answer contained a 0 in any of. the three columns to the/r^ight, 
it cannot be correct. ^ 

GO BACK TO PAGE NINETEEN AND TRY AGAIN. 



A 

\ 



Twenty-two 



That's correct; Very good! 



You realized that the trick is to "countV all those columns necessary 
to _add up to the decimal number seven, as illustrated below. 



. 2' 


■ 2' 


2^ 


2' 


2' 


2° 


(32) 


(16) 


(8) ^ 


(4) 


(2) 


(1) 


0 


0 ' 


0. 

\ 


i 


1 


, 1 



0 + 0 + 0 + 4 + 2 + 1 



7 Decimal Value 



In essence, the calculationa were as follows: 

.0 



2 X L = 1 
2^ X 1 = 2 
2^ X 1 = 4. 



r 



,1 + 2 + 4 = 7 \ , 

Since the decimal values of all tht^ee columns to the left are equal to 
MORE t'han seven, they could not Ke counted. Therefore, thej' get zeros. 
Of course, in a real situation, there is no need to put zeros in any 
column to the left of the one furthest to the left that counts. Those 
columns are simply left out. 

So, the decimal number 7 is represented in binary as 111, 



I 



GO ON TO PAGE TWENTY-SLX 



k'«. 



137 



Twenty-three 



If your answer contained a 1 in any of the three columns to • the left, 
it cannot be correct. 



GO BACK TO PAGE NINETEEN AND TRY AGAIN, 



1 



f 
I 



Twenty-four 



You seem to have misunderstood the meaning of the digits 0 and 1. 
This answer is incorrect, 

- ■ ^ V ) : 

GO BACK TO PAGE EIGHTEEN AND REREAD THE LAST PARAGRAPH. THEN TRY 
AGAIN. 



139 

O f7 



Twenfy-five 

You obviously have not been paving ^.attention . 
RE.READ PAGE EIGHTEEN ANd/tHEN TRY AGAIN, 



I 



Twenty-six 



To summarize, the binary number system used only two digits — 0 and ' 

.1/ ^ . ■ ' 

The heed for only two digits makes the binary number system an ideal 
vehicle for representing information within the computer. VHiy 1 this 

so? • . , ' ..^ , , ' . \ " . 

Well, many of the elements of the computeT^^system contain coriroonents - 
which can "be described a^ being in either otie of two states. \For ex^ 
ample, the computer 's electrical; circuitry caV^be either ON.. o£\,OFF, 
And the magnetic tape used in a computer systemOcan be polarised in 
one of two ways — POSITIVE or NEGATIVE. 

Magnetic tape is used to store information in the computer system .that 
%s not being immediately used. The information, thus stored is in a 
sense *'filed away" until it is needed for *some future operation. 

Some computer systems store information away on magnetic discs in- 
stead of magnetic tapes. The advantage of a disc is that it is possible 
to get' to a piece of information in the middle of the disc directly. In 
contrast, if one wants to get a piece of information from the middle of 
a tape, it is necessary to unwind the tape until the desired spot is 
^reachedi^ : , • 

The two kindsyof ways of getting to in^oxmation — directly, or only by 
running through the entire "file** — are called respectively: 

RANDOM access' 

and . 

SERIAL ACCESS 



GO ON TO PAGE TWENTY- SEVEN . 



ERIC- 



141 ; 
139 



Twenty^seven 



In the space below, indicate which of the two kinds of storage materials 
illustrates RANDOM ACCESS and which illustrates SERIAL ACCESS. ^ ' 

magnetic discs are a'kind of - access, 

/magnetic tapes are a kind of access. 

WRITE. D0\^ YOUR ANSWER. THEN GO ON TO PAGE TWKNT^-EIGHT AND COMPARE YOUR 
ANSWER WITH OURS. ' 




Twenty-eight 



The correct answers are: 

discs — random access 
tapes — serial access 

GO ON TO PAGE TWENTY -N I NE . 



ERIC 



141 

143 



0 



ERIC 



Twenty-nine 

You have now completed the teaching sequences of this lesson. 

Now you are ready to take tlie self -test that follows. 

After answering all the questions and checking your answers against 
the answer key, you will have a^r.better idea of what parts of the lesson 
you have mastered. 

Once you find the questions you got wrong, go back and reread those 
pages that deal with the material in question. 

"* ■ I*- 

GO ON TO PAGE THIRTY AND TAKE THE SELF-TEST. 



144 



142 



Thirty 



SELF-TEST 



-4 



Answer each of the following questions. Theh check your answers with 
the answer key. 



1. Add the appropriate exponent to each column below to ex- 
press the value of that column in a number system based on 
the number 12. 



12 12 12 12 12 12 12 



2. What is the decimal valufe of thd following binary numbers? 

' BINARY DECIMAL 

001100 = 
100000 = 
OIOIOI = 

3. What is the binary value of the decimal numbers below? 

DECIMAL BINARY 

27 = 
11 = 

' ,2 ^ = 

4. Describe' in your o\m words the special relationship between 
the octal and binary number systems, 

5. What is the decimal value of the following octal numbers? 

OCTAL BINARY 
. , 35 = 
20 
7 =, 



145 



Thrlty-one 



Why is the binary number system a good way to represent 
information within the computer? 

What decimal number does 8^ equal? 



1 



146 



Thirty-two 



ANSWER KEY 



12 



12' 



12 



12 



12 



t. 12 12 

2. DECIMAL . 

12 
32 

21 ■ . . 

3. BINARY 

OllOll * 

OOlOU 

000001 

4. Three binary digits translate easily into ai octal digit, 

-0 

5 . . DECIMAL . , . 

29 • 
16 

7 , 

6. ° ON/OFF is easily represented with two digits. 

7. 8° = 1 - - 



147 

145 



Thirtyr-three 



LIST OF POTENTIAL PROBLEMS 



GENERAL xCOMMENTS 



1. 



3; 



5. 

6." 

7. 
8. 

9. 

10. 



Title, objectives stated, and test are inconsistent' 

with text: octal is riot treated. 

\ ■ • . ' ' 

The. text contains a digression irrelevant to objectives 
as stated and to test: discussion of tape vs. disc 
storage. «. 



At least one questionable assumption: 
know about exponents and powers. 



that students 



No attempt made to individualize/differentially route 
students (who are likely to have heterogeneous back- 
grounds in math). 

Type face single spaced seems to be a bit hard to read. 

There is at least one long stretch with no interaction - 
that means no way of telling if the student has mastered 
the material, or is even paying attention. 

The title page is rather unattractive. 

The layout wastes paper. 

Not enough practice in the skills necessary to respond 
correctly to test items. There seems to be a need for 
some drill and practice segments. 

There is no differential remediation when the student 
answers incorrectly (except in one instance, when it is 
very unsatisfactory — see specific comments). 



LEVEL I 
LEVEL 1 

LEVEL 1 
.LEVEL 2 

LEVEL 4 
LEVEL 3^ 

LEVEL 4 
LEVEL 4' 
LEVEL 2/3 

LEVEL 3 



SPECIFIC COMMENTS 



Four 

Five,* Six 
Seven 

Eight ,*^ine 



Comment 

Of 

Reading level seems much too high 
Too much exposition without interaction 
Assumption that S knows exponents 
Ditto lack of .interaction 



Level 
LEVEL 3 

(//6 - gen. com.) 
(//3 - gen. com.)' 
(//6 ~ gen. com.^) 



ERLC 



148 



146 



Thirty-four 



Nino 
Nine 
Nine 

Ten, Twelve 

Ten, Twelve 
Fourteen 

Sixteen 

Sixteen 
Sixteen 

Seventeen 

Eighteen 

Eighteen 
Eighteen 
Nineteen 

Twenty 

Twenty 

Twenty 
•Twenty-one 



Comment ' ^ - * 

New concept introduced at the tail end of 
the frame 

Example buried in mound of text in last 
paragraph . 

Explanation unclear in last paragraph 

No attempt to individualize on basis of the 
student's response 

The "rule" is never made explicit 

Instructions telling' the student what to 
do. are unclear ^ . 

>-. 

Instructions telling the student what to 
do are unclear 

Labeling is awkward 

Poor question: the distractors give it 
away since they are so obviously wrong 

No remediation if the student doesn't 
understand 

Abrupt transition to different material 
between paragraphs 1 and 2 and between 
2 and 3 

No examples provided 

The label "decimal value" is missing 



Hug^e jump between pages eighteen and nine- 
teen as far as skills involved 

The choices are nonexclusive XI and 2 over- 
lap 4).. 

No chance for the student to say "l don't 
know," 

Choices are awkwardly worded 

No explanation as to why that is wrong. No 
help . 



Level 
LEVEL 3 

LEVEL 3 

LEVEL 3 

(#4 - gen, com.) 

LEVEL 3 
LEVEL 3 

LEVEL 3 

LEVEL 3 
LEVEL 3 ' 

(#10 - 'gen, com.) 
LEVEL 3 

LEVEL 3 
LEVEL 3 
LEVEL 3 

LEVEL 3 

LEVEL 3 

LEVEL 3 

(#10 - gen. com.) 



149 



147 



Thirty-five 



Twenty-one 
twenty- two 



Twenty- two 
Twenty- two 

Twenty- three 

Twenty-four 

Twenty-five 

Twenty-six 

Twenty-six 



Test 

Ql 
02 

Q3 

04 
Q5 



Comment . , ' \ 

Simple recycling the student back to the 
same question with no help is frustrating, 

y '■ 

Putting' the explanation of how to get the 
correct answer as feedback to the correct 
answer is rather pointless. The poor student 
really needs it when he is wrong. As a 
review, this is not a bad Idea, but as the 
only teaching sequence on the point, it is 
poorly placed. 

The explanation of the correct answer Is 
unclear^ — ^ 

Including new information in the feedback is 
not usually a good idea since students are 
not expecting it there and tend not to pay 
as much attention. * - 

Ditto Twenty-one ^ 

Ditto Twenty-one ' ^ , 

Ditto Twenty-one r 

This is a digression i 

The same frame is being used for two dis- 
tinct purposes (review of preceding concept 
and introduction of a new ohe) with no sig- 
nal to the student when he comes to thie 
bridge between the two. 

Comment 

The words "the value" are repeated. 

Student has no practice using this skill 
in the lesson, - 

Student has had no practice using this skill 
at this level of complexity. 

This- material is not covered in the lesson. 

Ditto Q2 

148" 



Level 

(//ID - gen, com.) 



LEVEL 3 



LEVEL 3 



LEVEL 3 



(//lO - gen , com. ) 
(//lO - gen. com. ) 
(#10 - gen. com.) 
(//2 - gen, com.) 
LEVEL 3 



Level 
LEVEL 4 

(//9 - gen. com. ) 

(#9 - gen. com.) 

(#2 - .gen . com, ) 
(#9 - gen com.) 



150 











... 
















! 




Thlrty-six 


0 




Tost, 


Cotranent 


Level 




(}b • 


Student was never requested to formulate 


(#9 - gen. com,) 


■ \ 
\ 


a response to this before. It was merely 








briefly presented ^ 








Covers point just mentioned in passing. 


LEVEL 3 






Students are likely not to recall it. 










J' 


• 






f 






* 


" \ ■ - ' 






















■ 








149 









151 
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21 - DISCUSSION 



The preceding segmeht of "rigged" programmed instruction 
was used by Ms, Jlerle /Goldstein in 1974 to test the hypothesis 
that reviewing skills could be taught.* Thirty-two students 
eni^oU^d in, Educational Psychology 211 participated in the ex- 
^i)«lMnt. Be^se Educational Psychology by Anderson and Faust 
(1973) was an/assigned text for this class » the experimenter 
assumed the^ students had some familiarity with the concepts 
to be ejc^ined by a review, Seve^^teen subjects (experimental 
group)/read some introductory materials giving the background 
and yrationale for lesson reviews and then studied a set ^f re- 
view guidelines differing only slightly from those found in 
/chapter il. The other subjects (control group) read a paper dis- 
cussing the PLATO and TICC^IT systems. At the end of the session 
for both groupa> thi^ materials were briefly discussed and handed 
in. At a second session all students were given the practice re- 
view segment tcT- critique O^^ly the students in the experimental 
grcup were given the guideline s to use during the sess^pn. The 
.criticisms made by the students Were placed on the programmed in- 
seduction booklet and/or bn blank paper. All materials were 
gathered and scored "blind'* Independently by two scorers. All 



*Some Guidelines for the Review of Itigtructural Materials by 
Merle Goldstein* 1974 » University of Illii|i9is (unpublished Mas^ . 
• ter's thesis). ' V, - 



153. 

• ' ~ ' - , i'5o 



comments except frame responses were classified as: 

- ■ ■ ■ ■ : ' ■ ■ ] ■ • ■ ' ■ 

Category A: StructJ^ural comments. /Level 1, 2, and 3 con- 
side ra\t ions.- 

' . ' ■ * 1 . , ■ 

Category B: Editorial commen*:s. Level 4 considerations/' 

" . ■ » • i • ■ • , . ' ' . ■ • 

c- , - i ■ 

Category C: Miscellaneous comments 

Though scoring was necessarily subj ective , '^'nt'er scorer re- 
liabilities'^ for categories A, B, and C w^re .90, .94, anc^>9^. 
Each student's score fojr each cat^egory was found by averaging 
the tptal number of comments the scorers found. Goldstein pro- " 
duced the folLowii^g table x>t results: ^ \ 

, Table I . ^ 

Sumifiary Data by Category and by Group ^ -J^.^^; • 
Structural Editorial Miscellaneous 



Condition ; 

Experimental. 0 
(n=17) . ' 

Control 



M- SD 
7.53 4.42' 

3.23 3.25 



M SD 
1.74 . 1.84 

0.13 ,0.30 



M _SD 
3.38 3.52 

4.03 4.96 



, Analysis .pf variance indicates the superiority af the students 
in the experimental group in categories A and B was significant 
at the f'005 level.. There was no significant difference in the num- 
ber of category C (miscellaneous) comments reeorded. Goldstein, 
observed that though students in the control group ^were extremely 
vocal in their disapproval of the lesson, they; were apparently un- 
. able to focus on specific problems or to channel their, feeling . 
into constructive criticism" of the lesson. 



ERIC 



/ 



I I5i 

L-"l54 



